UNIVERSIDADE DE SÃO PAULO ESCOLA DE ARTES, CIÊNCIAS E

HUMANIDADES PROGRAMA DE PÓS-GRADUAÇÃO EM SUSTENTABILIDADE

CELSO BARBIÉRI JÚNIOR

Abelhas e Ciência Cidadã: A utilização de abordagens participativas como

promotoras da Sustentabilidade

SÃO PAULO
2023
 CELSO BARBIÉRI JÚNIOR

Abelhas e Ciência Cidadã: A utilização de abordagens participativas como

promotoras da Sustentabilidade

Tese de Doutorado apresentada à Escola de

Artes, Ciências e Humanidades da
Universidade de São Paulo como requisito
para a obtenção do título de Doutor em
Ciências pelo Programa de Pós-Graduação em
Sustentabilidade. Área de concentração:
Ciência, Tecnologia e Gestão para a
Sustentabilidade Linha de pesquisa: Gestão
Ambiental Orientador(a): Prof. Dr. Tiago
Mauricio Francoy

São Paulo
2023

1
Autorizo a reprodução e divulgação total ou parcial deste trabalho, por qualquer meio convencional ou
eletrônico, para fins de estudo e pesquisa, desde que citada a fonte.

Ficha catalográfica elaborada pela Biblioteca da Escola de Artes, Ciências e Humanidades,

com os dados inseridos pelo(a) autor(a)
Brenda Fontes Malheiros de Castro CRB 8-7012; Sandra Tokarevicz CRB 8-4936

Barbiéri-Junior, Celso
Abelhas e Ciência Cidadã: A utilização de
abordagens participativas como promotoras da
Sustentabilidade / Celso Barbiéri-Junior;
orientador, Tiago Mauricio Francoy. -- São Paulo,
2023.
154 p: il.

Tese (Doutorado em Ciencias) - Programa de Pós-

Graduação em Sustentabilidade, Escola de Artes,
Ciências e Humanidades, Universidade de São Paulo,
2023.
Versão corrigida

1. Meliponicultura. 2. Abelhas sem Ferrão. 3.

Avaliação de Aprendizado. 4. Ciência Participativa.
5. Monitoramento de biodiversidade. 6. Conservação
Biológica. I. Francoy, Tiago Mauricio, orient. II.
Título.
Nome: BARBIÉRI, Celso Jr.
Abelhas e Ciência Cidadã: A utilização de abordagens participativas como promotoras da
Sustentabilidade

Tese de Doutorado apresentada à Escola de

Artes, Ciências e Humanidades da
Universidade de São Paulo como requisito
para a obtenção do título de Doutor em
Ciências pelo Programa de Pós-Graduação em
Sustentabilidade. Área de Concentração:
Ciência, Tecnologia e Gestão para a
Sustentabilidade

Aprovado em: ___ / ___ / _____

Banca Examinadora

Prof. Dr. ____________________ Instituição: __________________

Julgamento: ____________________ Assinatura: __________________

Prof. Dr. ____________________ Instituição: __________________

Julgamento: ____________________ Assinatura: __________________

Prof. Dr. ____________________ Instituição: __________________

Julgamento: ____________________ Assinatura: __________________

Prof. Dr. ____________________ Instituição: __________________

Julgamento: ____________________ Assinatura: __________________

Prof. Dr. ____________________ Instituição: __________________

Julgamento: ____________________ Assinatura: _________________

3
 Dedico este trabalho a todos aqueles que, em meio a todas as adversidades,
dedicam-se à defesa e promoção da ciência, educação e conservação do meio
ambiente.

4
 AGRADECIMENTOS

Aqui tenho de agradecer a incontáveis pessoas e aqui o faço nominalmente

para algumas.
A minha companheira, Paula Drago, por todo apoio, companheirismo e
engajamento ativo no projeto BeeKeep.
Ao meu orientador, Tiago Francoy, pela orientação, amizade e incentivo para
desenvolver este projeto.
Aos companheiros de pesquisa em Ciência Cidadã, Sheina Koffler (em
especial, por coidealizar todo o projeto), Jailson Leocádio, Natalia Pirani
Ghilardi-Lopes, Bruno Albertini e Antônio Mauro Saraiva que são corresponsáveis
pelo sucesso de tantas empreitadas em nossos projetos de pesquisa e extensão.
A Michael Pocock, pela receptividade e todos os ensinamentos sobre ciência
cidadã e redes sociais no meu período de intercâmbio no Reino Unido.
Aos amigos do Meliponicultura.org, André de Matos Alves e Maria Gabriela
Silva, por toda a ajuda durante todos esses anos de empreitada na Meliponicultura.
A todos os ministrantes e participantes de todas as edições do Curso de
Extensão em Meliponicultura e Ciência Cidadã da Universidade de São Paulo, todos
os cientistas cidadãos engajados nos projetos da Plataforma BeeKeep e a todos os
meliponicultores que responderam ao questionário do Censo da Meliponicultura de
2022.
Aos meus colegas do LABBEES, pelo coleguismo, apoio e tornar a jornada
mais divertida, em especial a Lucas Paulino dos Santos, Fernanda Assumpção
Costa, Gabriel Reginato Silva e Luana Ferreira Fedele, que comigo produziram
materiais para a plataforma BeeKeep.
As professoras Wânia Duleba e Blandina Viana pelas contribuições feitas no
meu exame de qualificação.
Agradeço a CAPES pelo financiamento da minha bolsa de doutorado,
número do processo:88887.606651/2021-00, bem como a bolsa CAPES-PRINT,
número do processo:88887.372798/2019-00, que me possibilitaram concluir mais
uma etapa da minha formação. Agradeço a FAPESP pelo financiamento do projeto
Salvaguardando serviços de polinização em um mundo em mudança: teoria na
prática (SURPASS2), número do processo 2018/14994-1, que viabilizou a
construção desta pesquisa.

5
 RESUMO

BARBIÉRI JÚNIOR, Celso. Abelhas e Ciência Cidadã: A utilização de abordagens

participativas como promotoras da Sustentabilidade.2023 140p. Tese (Doutorado em
Ciências) - Escola de Artes, Ciências e Humanidades, Universidade de São Paulo,
São Paulo, 2023. Versão original.

Pesquisas em Ciência Cidadã com abelhas vem se tornando cada vez mais
populares ao redor do mundo, embora ainda haja uma grande concentração de
trabalhos no hemisfério norte. Algumas iniciativas despontam em países em
desenvolvimento como o Brasil, representando oportunidades de implementar
diversos Objetivos para o Desenvolvimento Sustentável. Nesse contexto,
desenvolvemos um protocolo de monitoramento de atividade de voo de abelhas
operárias dentro da plataforma BeeKeep, o qual se apresenta como um projeto
bastante viável e acessível para o monitoramento de abelhas sem ferrão, que
ocorrem majoritariamente no sul global. Tal protocolo demonstrou alta confiabilidade
de dados para mensurar a entrada e saída de abelhas dos ninhos de abelhas sem
ferrão, ao comparar o monitoramento realizado por cientistas cidadãos com o devido
treinamento com cientistas profissionais especialistas em abelhas sem ferrão. A
avaliação da qualidade dos dados, demonstrou-se particularmente importante para
o aprimoramento da metodologia, principalmente em relação ao monitoramento de
abelhas entrando com cargas de pólen nas corbículas. Conclusão relevante deste
trabalho é a demonstração de que o treinamento e a avaliação da participação de
cientistas cidadãos são de grande importância para a obtenção de resultados
robustos e confiáveis. Nesse sentido, o Curso de Extensão em Meliponicultura e
Ciência Cidadã da Universidade de São Paulo demonstrou considerável sucesso
em diferentes frentes, proporcionando conteúdo educacional gratuito sobre abelhas
e meliponicultura, capacitando os cientistas cidadãos para executar o protocolo de
proposto e gerar aprendizado significativo para os participantes, tanto em aspectos
de autoefícacia, conhecimento sobre processos científicos, natureza do
conhecimento científico e conteúdo científico relacionado a abelhas e
meliponicultura. Um dos indicadores do sucesso do curso ao gerar aprendizado é
que, ao avaliar a influência de características do perfil dos participantes do curso,
mesmo havendo diferenças significativas em seus scores iniciais, dependendo do
perfil, houve aprendizado significativo em todos os grupos, havendo inclusive maior
aprendizado em grupos que apresentaram menores scores na avaliação inicial.
O presente trabalho traz resultados promissores que confirmam o potencial
da linha de pesquisa pioneira na interface entre meliponicultura e ciência cidadã
devido a fatores como a qualidade dos dados gerados bem como o aprendizado e
interesse por parte dos cientistas cidadãos que ganham autonomia enquanto
produtores de conhecimento e promotores da sustentabilidade.

Palavras chave: Meliponicultura; Abelhas sem Ferrão; Avaliação de

Aprendizado; Ciência Participativa; Monitoramento de biodiversidade.

6
 ABSTRACT

BARBIÉRI JÚNIOR, Celso. Bees and Citizen Science: The use of participatory
approaches as promoters of Sustainability.2023 140p. PhD Thesis (Postgraduate
Program in Sciences) - School of Arts, Sciences and Humanities, University of São
Paulo, São Paulo, 2023. Original version.

Citizen Science research on bees is becoming increasingly popular around the

world, although there is still a large predominance of projects in the northern
hemisphere. Some initiatives emerge in developing countries such as Brazil,
representing opportunities to implement several Sustainable Development Goals. In
this context, we developed a protocol for monitoring the flight activity of worker bees
within the BeeKeep platform, which presents itself as a very viable and accessible
project for monitoring stingless bees, which occur mainly in the global south. This
protocol demonstrated high reliability of data to measure the entry and exit of bees
from stingless bee nests when comparing the monitoring carried out by citizen
scientists with appropriate training with professional scientists specialized in
stingless bees. The evaluation of the data quality proved to be particularly important
for the improvement of the methodology, mainly in relation to the monitoring of bees
entering pollen loads attached to the pollen baskets. A relevant conclusion of this
work is the demonstration that training and evaluating the participation of citizen
scientists are of great importance for obtaining robust and reliable results. In this
sense, the University of São Paulo’s Extension Course in Meliponiculture and Citizen
Science demonstrated considerable success on different fronts, providing free
educational content about bees and meliponiculture, training citizen scientists to
execute the proposed protocol, and generating significant learning for participants,
both in aspects of self-efficacy, knowledge about scientific processes, nature of
scientific knowledge and scientific content related to bees and meliponiculture. One
of the indicators of the course’s success in generating learning is that, when
evaluating the influence of profile characteristics of the course participants, even with
significant differences in their initial scores, depending on the profile, there was
significant learning in all groups, with even greater learning in groups that had lower
scores in the initial assessment. The present work brings promising results that
confirm the potential of the pioneering line of research in the interface between
meliponiculture and citizen science due to factors such as the quality of the data
generated, as well as the learning and interest on the part of citizen scientists who
gain autonomy as producers of knowledge and sustainability promoters.

Keywords: Meliponiculture; Stingless bees; Learning evaluation; Participative

Science; Biodiversity Monitoring.

7
 SUMÁRIO

1 - INTRODUÇÃO 9
2 - OBJETIVOS e ORGANIZAÇÃO GERAL DA TESE 18
3 - CAPÍTULO I “Abelhas e Polinização” 22
4 - CAPÍTULO II “A Buzz for Sustainability and Conservation: The Growing Potential
of Citizen Science Studies on Bees” 28
5 - CAPÍTULO III “Theoretical model for interdisciplinary analysis of human activities:
Meliponiculture as an activity that promotes sustainability” 60
6 - CAPÍTULO IV Data Reliability in a Citizen Science Protocol for Monitoring Stingless
Bees Flight Activity 85
7 - CAPÍTULO V “Evaluation of a Bee-focused Citizen Science Project: Influences of
participants’ profile in their learning” 113
8 - CONCLUSÃO 150
9 - REFERÊNCIAS 153

8
1 - INTRODUÇÃO

Organismos polinizadores possuem um papel fundamental na manutenção

da biodiversidade mundial, reconhecido a partir da aprovação da Convenção da
Diversidade Biológica, da qual o Brasil é signatário. A convenção foi parte da
Conferência das Nações Unidas sobre Meio Ambiente (Eco-92) realizada no Rio de
Janeiro, em 1992. O texto da Convenção Sobre da Diversidade Biológica foi
transformado em decreto em 1994 (BRASIL 1994). Posteriormente, em 1998 foi
realizada uma reunião com especialistas em polinizadores do mundo todo, que teve
como resultado o documento Declaration on Pollinators apresentado à Convenção
da Diversidade Biológica, para a implantação de uma Iniciativa Internacional dos
Polinizadores (IPI). No mesmo ano também foi criada a Iniciativa Brasileira dos
Polinizadores, como maneira de reforçar a importância destes organismos e
fornecer uma melhor gestão da criação e compartilhamento do conhecimento
(IMPERATRIZ-FONSECA et al 2007).
No Brasil, as abelhas são responsáveis pela polinização de cerca de 80%
das plantas cultivadas ou silvestres utilizadas na alimentação, sendo as
polinizadoras exclusivas de 65% destas espécies (BPBES, 2019). O valor
econômico global da polinização foi calculado pelo Relatório de Avaliação sobre
Polinizadores, Polinização e Produção de Alimentos da IPBES (Intergovernmental
Science-Policy Platform on Biodiversity and Ecosystem Services) , sendo estimado
entre US$ 235 bilhões e US$ 577 bilhões ( POTTS et al. in IPBES 2016). Estima-se
para o Brasil, que a polinização relacionada à produção agrícola tem um valor anual
de R$ 43 bilhões para 2018 (BPBES, 2019). Nesse sentido, as abelhas demonstram
enorme importância ecológica e econômica, tanto para a conservação da maior
parte das espécies vegetais nativas, quanto para a utilização em programas de
polinização aplicada a cultivos agrícolas (MESQUITA, 2009), de modo que
necessitam de uma atenção especial no campo da conservação (SILVEIRA et al.,
2006).
A região Neotropical possui uma melitofauna extremamente rica (MOURE et
al. 2007), porém estima-se que somente um terço de toda sua diversidade seja
conhecida. Existem grandes lacunas em informações sobre riqueza, diversidade,
distribuição e impactos de atividades humanas, além da existência de regiões nas

9
quais as coletas foram realizadas de maneira insuficiente ou que não foram
amostradas ainda (FREITAS et al., 2009).
Dentre os diversos grupos de abelhas, encontramos a tribo Meliponini, cujas
espécies componentes são conhecidas popularmente como abelhas indígenas sem
ferrão, por possuírem um ferrão vestigial atrofiado, sem capacidade de ferroar. Elas
são responsáveis pela visitação e potencial polinização de até 90% das plantas de
determinadas regiões, sendo assim, fundamentais para a conservação de áreas
nativas e também de agroecossistemas (Kerr et al., 1996).
As abelhas sem ferrão, usualmente constroem seus ninhos em cavidades
previamente existentes, como ocos de árvores, formigueiros ou cupinzeiros
abandonados, sendo que espécies com hábito de nidificação em colmeias externas
representam uma minoria (Nogueira-Neto 1997). Com uma grande diversidade de
características morfológicas e comportamentais, a tribo Meliponini. São mais de 600
espécies válidas no mundo, sendo que aproximadamente 400 ocorrem nas
Américas e cerca de 300 no Brasil, (GRÜTTER et al., 2020).
As abelhas sem ferrão encontram-se em processo acelerado de diminuição e
desaparecimento de suas populações, provocado principalmente pela destruição de
matas nativas, ambiente preferencial de nidificação da grande maioria dessas
espécies (LOPES et al., 2005; BROWN & ALBRECHT, 2001). Kerr et al. (2010)
também apontam os desmatamentos, queimadas e a expansão das zonas urbanas
em regiões que antes eram de florestas nativas como causadores da redução da
biodiversidade de polinizadores, por impactar sobre seus recursos alimentares e
sítios de nidificação. Os mesmos autores ainda destacam que, no Brasil, muitas
espécies de abelhas nativas estão gravemente ameaçadas de extinção pelas ações
supracitadas e pelo uso indiscriminado de agrotóxicos.
Nesse contexto de perda da biodiversidade e esforços para diminuir tal
processo, mecanismos participativos, como a ciência cidadã, vem demonstrando
grande sucesso enquanto medida em prol da conservação (MACPHAIL & COLLA,
2020). A Ciência Cidadã pode ser definida como o envolvimento ativo do público
geral em atividades científicas para produzir novos conhecimentos científicos
(BONNEY et al, 2016). De acordo com SILVERTOWN (2009), um cientista cidadão
é um voluntário que coleta ou processa informações de parte de uma pesquisa
científica. Embora observações de naturalistas amadores venham sendo
importantes há séculos, projetos com ciência cidadã (citizen science) vêm se

10
popularizando com maior intensidade desde a década de 2000, devido a
possibilidade de acompanhar impactos sociais e ecológicos de grande escala
através da internet (LEPCZYK et al. 2009). Atualmente cientistas amadores e
profissionais têm acesso a um crescente número de ferramentas para explorar
mudanças na fenologia, distribuição, abundância relativa, sucesso reprodutivo de
organismos ao longo do tempo e do espaço. Neste processo, a utilização de ciência
cidadã tem influenciado tanto a escala das pesquisas, quanto a relação entre os
pesquisadores e o público (DICKINSON et al 2010).
Esses projetos vêm ganhando adesão da população pelo mundo todo,
possibilitando a obtenção de cada vez mais dados sobre a biodiversidade global,
bem como aproximando os cidadãos da construção do conhecimento sobre
biodiversidade. De acordo com (DUNN et all, 2016) insetos oferecem maiores
oportunidades para os cientistas cidadãos do que a maior parte dos grupos
biológicos. Isso se deve, em geral, ao seu tamanho, que permite que sejam
fotografados, por terem a sua coleta permitida (na maioria dos casos), além da
grande quantidade de assuntos pouco estudados em relação a sua diversidade (e
consequentemente quanto a suas interações ecológicas). Uma forma de incorporar
a utilização de insetos em projetos de ciência cidadã é semelhante à utilizada com
aves, através da coleta de amostras não destrutivas (ex: fotografias) para, assim,
determinar a distribuição e abundância de espécies de interesse, e em alguns
casos, como as populações se alteram ao longo do tempo.Um exemplo bem
sucedido de projeto envolvendo ciência cidadã e insetos foi realizado por
pesquisadores do Japão, que entre 2006 e 2015 estudaram abelhas do gênero
Bombus, através de fotografias coletadas e fornecidas por voluntários de todo o
Japão, obtendo como principais resultados um maior engajamento da população
com a produção de ciência e uma estimativa da distribuição e fatores que
influenciam a distribuição de seis espécies de abelhas (Suzuki-Ohno et al, 2017).
Devido a importância ecológica, necessidade de conservação e problemas na coleta
de informações sobre sua diversidade, distribuição e fenologia, as abelhas sem
ferrão são um bom alvo para projetos de ciência cidadã.
A implementação de projetos de ciência cidadã com grupos previamente
interessados abre a possibilidade da aplicação de alguns protocolos mais
elaborados e da obtenção de dados que ao mesmo tempo dificilmente seriam
coletados por usuários sem conhecimento prévio do assunto, ou que teriam maior

11
necessidade de aprendizado para executar tais protocolos, potencializando o
alcance da coleta de dados, que levaria muito mais tempo e recursos ao ser
executada apenas por especialistas (POCOCK et al, 2014). Dessa forma, um
projeto de ciência cidadã com abelhas sem ferrão possui grande potencial para a
coleta de dados ecológicos e demonstra-se uma ferramenta interessante para
compreender a dinâmica das interações entre o público geral e a academia, além de
chamar atenção para a causa da necessidade de conservação das abelhas e
importância dos polinizadores.
Outra vantagem na escolha da tribo Meliponini como alvo de um projeto de
ciência cidadã é o engajamento prévio pelos meliponicultores, praticantes da
meliponicultura, a criação racional de abelhas sem ferrão (Venturieri et al 2007,
Nogueira -Neto 1954). De acordo com Barbiéri, 2018, embora a meliponicultura seja
uma atividade com alto potencial econômico, pela produção e comércio de mel e
colônias de abelhas, entre as múltiplas motivações que levam os meliponicultores a
criar abelhas sem ferrão, encontram-se a vontade de ajudar a conservar essas
espécies (84,64%) e o lazer proporcionado (66,07%) . Além disso, a meliponicultura
apresenta uma crescente organização e associativismo, tornando os
meliponicultores elementos chave na realização e disseminação de iniciativas
participativas para a conservação de abelhas nativas, incluindo projetos de ciência
cidadã.
Visando ampliar o conhecimento popular e científico sobre abelhas sem
ferrão aliada a conservação deste grupo de polinizadores e desenvolvimento
sustentado da meliponicultura foi concebida a plataforma de ciência cidadã
BeeKeep. A plataforma funciona como uma aplicação online, que pode ser
acessada por dispositivos móveis (tablets e smartphones) ou computadores.
As funções da plataforma estão sendo desenvolvidas e implementadas em
fases, seguindo as especificações feitas pelo grupo de pesquisa. A seguir temos um
breve descritivo das funções da plataforma e seus objetivos no contexto geral do
projeto.O primeiro protocolo a ser implementado na plataforma BeeKeep, foi o
“Protocolo de Monitoramento de Atividade de Voo de Abelhas sem Ferrão”, em 2020
no contexto do Curso de Extensão em Meliponicultura e Ciência Cidadã da
Universidade de São Paulo.

12
 O protocolo encontra-se disponível em Beekeep PCS (beekeep.pcs.usp.br),
e nele, usuário cientista cidadão deve executar um protocolo estruturado de
contagem de abelhas em atividade de voo (Koffler et al., 2021). Além de
informações sobre o local, data, hora e condições do tempo no momento da coleta
de dados, é solicitado ao usuário que grave um vídeo de 30s da entrada de ninhos
de abelha sem ferrão e faça contagens nestes vídeos (do número de abelhas
entrando e saindo do ninho e do número de abelhas entrando no ninho portando
pólen). Além de submeter vídeos (funcionalidade gravação), os usuários também
podem realizar contagens em vídeos submetidos por outros cientistas cidadãos
(funcionalidade contagem). A partir do feedback dos usuários e da validação dos
dados produzidos na plataforma , diversas melhorias já foram realizadas na
plataforma.

Princípios norteadores da construção da Plataforma BeeKeep

Para assegurar as boas práticas em ciência cidadã nos projetos correlatos a

plataforma BeeKeep, foram estabelecidas diversas medidas que estão sendo
cumpridas e foram implementadas no projeto e trabalhos derivados, incluindo os
artigos que compoem a presente tese. Para tanto, foram utilizados como referencial
teórico as estratégias empregadas nos trabalhos avaliados na revisão sistemática
sobre ciência cidadã e abelhas apresentada no capítulo II desta tese, bem como nos
dez princípios da ciência cidadã, de acordo com a ECSA (European Citizen Science
Association).
Princípio 1 “Os projetos de ciência cidadã envolvem ativamente os cidadãos
em atividades científicas que gerem novos conhecimentos ou compreensão”:
Os cientistas cidadãos participam e participarão ativamente do projeto,
levantando perguntas, coletando e ajudando a analisar dados que serão revisados e
publicados em periódicos científicos.
Princípio 2 “Projetos de ciência cidadã têm um resultado científico genuíno”:
Desde as primeiras etapas de sua concepção, a plataforma Beekeep
busca responder perguntas científicas relevantes em diversas áreas do
conhecimento, como ecologia, educação, sociologia, ciências ambientais, ciência de
dados, entre outras.

13
 Princípio 3 - “Tanto os cientistas profissionais quanto os cientistas cidadãos
se beneficiam com a participação”:
Como já destacado, é de extrema importância para o projeto que os
voluntários também sejam beneficiados com sua participação, dessa forma a
plataforma foi projetada para contribuir com o aprendizado dos usuários em
questões relativas a abelhas, plantas, polinização e criação de abelhas. Além disso,
os meliponicultores serão beneficiados diretamente pelo uso das ferramentas ainda
não implementadas de monitoramento de recipientes-isca e do prontuário de
manejo, facilitando suas práticas cotidianas.
Princípio 4 - “Os cientistas cidadãos podem, se desejarem, participar de
vários estágios do processo científico”:
Os cientistas cidadãos foram incluídos em múltiplos estágios do processo
científico, desde a concepção do projeto, uma vez que vários meliponicultores foram
consultados sobre ferramentas úteis para a atividade, e o projeto nasceu partindo de
uma demanda da própria comunidade. Além disso, os voluntários participam da
coleta, e auxiliam na análise de dados, como demonstrado no capítulo IV da
presente tese. O projeto oriundo do Protocolo de Monitoramento de Atividade de
Voo foi nomeado a partir de sugestões de cientistas cidadãos voluntários do projeto
a partir de uma votação virtual com a comunidade de participantes, chegando ao
nome de “Projeto #CidadãoASF”.
Princípio 5 “Cientistas cidadãos recebem feedback do projeto”:
Os cientistas cidadãos recebem feedback sobre o projeto de diversas formas,
entre elas as atualizações sobre seus registros, visualização dos dados coletados,
informações sobre os estudos no site e nas mídias sociais do projeto. Por meio dos
canais de comunicação da plataforma, cientistas profissionais e voluntários mantém
contato sobre o o funcionamento da plataforma, andamento dos projetos e para
sanar duvidas que eventualmente surgem no decorrer do projeto, tendo o feedback
dos voluntários sido de grande valia para o aperfeiçoamento da plataforma, quanto
dos protocolos em curso.
Princípio 6 “A ciência cidadã é considerada uma abordagem de pesquisa
como qualquer outra, com limitações e vieses que devem ser considerados e
controlados”:
Conhecendo as possíveis limitações das abordagens de ciência cidadã,
foram estabelecidas diversas medidas para assegurar a qualidade de dados.

14
a) Protocolos: para todas as pesquisas realizadas com a plataforma
foram ou estão sendo desenvolvidos protocolos, visando facilitar e
simplificar as atividades realizadas pelos cientistas cidadãos, bem
como reduzir vieses e erros. Isso inclui, no caso do protocolo de
monitoramento de atividade de voo, o fornecimento de um guia com
todas as informações necessárias para a execução do protocolo com
boas práticas;
b) Treinamento - foram e estão sendo desenvolvidos materiais de apoio
para a identificação de abelhas sem ferrão, boas práticas em
meliponicultura e serão feitos guias específicos para instruir os
usuários sobre o uso da plataforma, materiais informativos vem sendo
elaborados pelo grupo de pesquisa, e um curso completo de
meliponicultura, sustentabilidade e ciência cidadã já foi ministrado e
disponibilizado aos interessados, podendo este ser acessado a
qualquer momento no canal do YouTube do Meliponicultura.org,
coletivo parceiro do projeto
(https://www.youtube.com/@Meliponicultura);
c) Equipamento calibrado - a aplicação online utiliza dados e metadados
fornecidos pelos dispositivos móveis de forma a garantir a precisão
das medidas;
d) Comprovantes digitais - os registros contam com fotos ou vídeos, de
acordo com o protocolo, que ficam armazenados no banco de dados
do projeto, hospedado na nuvem da Universidade de São Paulo;
e) Conhecimento pessoal - especialmente nos projetos relacionados a
meliponicultura, contamos com a experiência e conhecimento de
meliponicultores e apicultores para ampliar a qualidade de dados, que
também podem colaborar muito com a identificação de espécimes de
abelhas sem ferrão;
f) Repetição de amostragem - em diversos protocolos a repetição de
amostragem é requerida ou recomendada, como no monitoramento de
recipientes-isca e prontuário de manejo, ou a contagem por diversos
cientistas cidadãos no protocolo de monitoramento de atividade de
voo;

15
 g) Filtragem de registros incomuns - contando com uma ferramenta de
visualização de dados, os registros incomuns são filtrados pelos
pesquisadores, os usuários também poderão informar registros
incomuns quando o percebem;
h) Revisão de especialistas - especialistas internos e parceiros do grupo
colaboram com a revisão de identificações/registros, aumentando a
confiabilidade dos dados;

Princípio 7 - “Os dados e metadados do projeto de ciência do cidadão são

disponibilizados publicamente e, sempre que possível, os resultados são publicados
em formato de acesso aberto”:
Os dados são disponibilizados online, respeitando a privacidade dos usuários
de forma que os cientistas cidadãos tenham acesso, além disso o grupo de
pesquisa busca publicar os artigos decorrentes em periódicos de acesso aberto ou
em periódicos restritos desde que o artigo esteja em acesso aberto.
Princípio 8 - Cientistas cidadãos são reconhecidos/agradecidos em
resultados de projetos e publicações:
O grupo de pesquisa se compromete a reconhecer e agradecer os
voluntários e partes envolvidas nas publicações decorrentes do projeto.
Princípio 9 - Os programas de ciência cidadã são avaliados por sua produção
científica, qualidade de dados, experiência do participante e impacto social ou
político mais amplo:
Entre as medidas para o cumprimento do princípio 9 estão:
a) Qualidade de dados - Como mencionado anteriormente, várias
medidas estão sendo tomadas para garantir uma maior qualidade de
dados, sendo este o objetivo central abordado no capítulo IV;
b) Experiência do participante - um dos objetivos do presente trabalho de
doutorado é avaliar a experiência do participante, em relação ao seu
aprendizado e motivações, sendo este o objetivo central abordado no
capítulo V;
c) Impacto social e político - as pesquisas derivadas do projeto Beekeep
visam colaborar com políticas públicas relacionadas aos setores de
educação, conservação biológica e fortalecimento da cadeia da
meliponicultura;

16
 Princípio 10 “Os líderes de projetos de ciência cidadã levam em
consideração as questões legais e éticas em torno dos direitos autorais, propriedade
intelectual, acordos de compartilhamento de dados, confidencialidade, atribuição e
impacto ambiental de quaisquer atividades”.

a) Compartilhamento e confidencialidade de dados - a plataforma Beekeep

respeita a Lei Geral de Proteção de Dados Pessoais (LGPD), Lei nº 13.709,
de 14 de agosto de 2018, não coletando informações sem o consentimento
do usuário que ao se cadastrar no projeto deve aceitar os termos de uso da
plataforma, bem como a sua política de privacidade, devidamente informada;
b) Atribuição e autoria - os autores dos dados submetidos na plataforma
continuam sendo donos dos dados, embora cedam a exclusividade de
direitos autorais dos mesmos ao submetê-los, conforme indicado nos termos
de uso;
c) Impacto ambiental - diversos cuidados foram tomados visando evitar
impactos ambientais negativos decorrentes do projeto, como a redução da
precisão da localização dos registros de colônias, de forma a evitar
depredações por pessoas mal intencionadas.

Curso de Extensão em Meliponicultura e Ciência Cidadã

Tendo a maior parte do presente trabalho de doutorado sido desenvolvido

durante a pandemia de COVID-19, e dada a impossibilidade da realização de
atividades presenciais, foram elaboradas alternativas para a condução de um
projeto de ciência cidadã sem a necessidade de encontros presenciais. Dessa
forma, ainda no primeiro semestre de 2020 foi estruturado um curso de extensão em
meliponicultura e ciência cidadã para o oferecimento em modalidade virtual. O curso
foi aperfeiçoado ao longo de quatro edições e contou com a participação de
pessoas de diversas regiões do Brasil.
O primeiro módulo é dedicado à biologia geral, importância ecológica e
diversidade de abelhas. O segundo módulo trata da meliponicultura como promotora
de sustentabilidade, técnicas de manejo de abelhas sem ferrão e do ambiente em
que estão inseridas. Já o terceiro módulo é direcionado a ciência cidadã, incluindo

17
fundamentos de metodologia científica, ciência cidadã, exemplos de ciência cidadã
com abelhas e uma atividade prática. os discentes do curso atuam como cientistas
cidadãos de duas formas. O primeiro grupo, formado por voluntários que têm
acesso a um ninho aplicam um protocolo de monitoramento da atividade de vôo de
uma colônia de abelhas sem ferrão por meio de uma aplicação para dispositivos
móveis, enquanto outro grupo atua na validação do protocolo, realizando a
contagem de abelhas que saem do ninho, entram no ninho, e entram no ninho
carregando pólen.
O curso é realizado de forma online e é utilizada a plataforma de aprendizado
virtual Moodle-USP como fórum para dúvidas. As videoaulas são disponibilizadas no
canal do Youtube da plataforma Meliponicultura.org. Adicionalmente, o curso conta
com sessões de “tira-dúvidas” com os docentes em formato live no Youtube,
respondendo às questões mais frequentes do fórum, bem como alguns tópicos
adicionais que emergem durante a discussão.
Desde o primeiro oferecimento, o curso obteve uma grande procura, indo ao
encontro do que é apontado por Jaffé et al, (2015), que apontam como duas das
principais dificuldades encontradas pelos meliponicultores a falta de conhecimento
geral, e dificuldade com técnicas de manejo. Houve alta demanda pelo curso em
cada uma das quatro edições, tendo sido o quarto oferecimento o maior curso de
extensão em número de inscritos (3913) e formados (1112) da história da
Universidade de São Paulo, segundo a própria Pró-Reitoria de Cultura e Extensão
da USP.

2 - OBJETIVOS e ORGANIZAÇÃO GERAL DA TESE

No contexto da perda de biodiversidade e declínio das populações de

abelhas o presente trabalho tem seus objetivos relacionados a dois grandes temas
que encontram-se aqui intimamente relacionados, a participação da população geral
em projetos científicos e a meliponicultura. Em relação a participação da população
em pesquisas, esta pode ocorrer de forma mais ativa como em um projeto de
ciência cidadã, ou menos diretamente, como na participação em uma pesquisa
estruturada, via questionário.

18
 Para cada objetivo, é dedicado um capítulo, apresentado na forma de artigo
científico ou capítulo de livro, no qual são apresentados seus contextos, objetivos
particulares, referencial teórico, métodos, resultados e conclusões.

O Capítulo I “Abelhas e Polinização”, apresentado em formato de capítulo de

livro e publicado no livro “Ciência Cidadã e Polinizadores da América do Sul” tem
como objetivos apresentar a importância das abelhas e polinização, bem como um
pouco sobre sua diversidade na América Latina, em que encontramos uma grande
diversidade de abelhas pertencentes a diferentes famílias, incluindo as abelhas
nativas sem ferrão, as abelhas das orquídeas e as mamangavas de chão. Além de
como o estudo dessas abelhas é relevante para a conservação da biodiversidade e
para práticas sustentáveis relacionadas à meliponicultura e apicultura.
Referência: BARBIÉRI, CELSO; FLORES-PRADO, L. ; FRANCOY, T. M. ;
GEISA, M. G. ; GENNARI, G. P. ; FALEIROS-QUEVEDO, M. . Abelhas e
Polinização. In: Ghilardi-Lopes, Natalia Pirani, & Zattara, Eduardo Enrique. (Org.).
Ciência cidadã e polinizadores da América do Sul. 1ed.São Carlos - SP: Editora
Cubo, 2022, v. 1, p. 39-45.

No Capítulo II “A Buzz for Sustainability and Conservation: The Growing

Potential of Citizen Science Studies on Bees”, apresentado no formato de artigo
científico, publicado no periódico “Sustainability” o objetivo é por meio de uma
revisão sistemática dos estudos de ciência cidadã sobre abelhas, avaliar como
esses estudos podem contribuir para o relato e monitoramento dos ODS, além de
verificar a adesão aos princípios da ciência cidadã. Para isso,foram coletados 88
estudos publicados no período de 1992 a 2020. Assim, avaliamos a literatura
científica que trata da intersecção entre os temas “abelhas” e ciência cidadã sob o
prisma dos dezessete Objetivos de Desenvolvimento Sustentável da Organização
das Nações Unidas (ONU) e a luz dos Dez Princípios da Ciência Cidadã da
Associação Européia de Ciência Cidadã “European Citizen Science Association”
(ECSA).
Referência: KOFFLER & BARBIERI et al. A buzz for sustainability and
conservation: the growing potential of citizen science studies on bees.
Sustainability, v. 13, n. 2, p. 959, 2021.

19
 O Capítulo III “Theoretical model for interdisciplinary analysis of human
activities: Meliponiculture as an activity that promotes sustainability” apresentado no
formato de artigo científico, publicado no periódico “Ambiente & Sociedade” propõe
um modelo teórico para a análise interdisciplinar de atividades humanas, utilizando
a meliponicultura como objeto de estudo para exemplificar o método, abrangendo os
domínios ambiental, cultural, social e econômico. Neste capítulo apontamos como é
fundamental que políticas públicas participativas e interdisciplinares sejam
implementadas para incentivar essa atividade, integrando os diferentes atores
envolvidos, como meliponicultores, cientistas, governos, empresas e organizações
não governamentais.
Referência: BARBIÉRI, Celso; FRANCOY, Tiago Mauricio. Theoretical model
for interdisciplinary analysis of human activities: Meliponiculture as an activity that
promotes sustainability. Ambiente & Sociedade, v. 23, 2020.

A proposta do Capítulo IV “Data Reliability in a Citizen Science Protocol for

Monitoring Stingless Bees Flight Activity” apresentado no formato de artigo
científico, publicado no periódico “Insects” é validar um protocolo de ciência cidadã
para monitorar a atividade de voo de abelhas sem ferrão. A contagem da atividade
de voo (entrada, saída e entrada com pólen) filmada em vídeos de 30 segundos foi
comparada entre três grupos diferentes: "cientistas cidadãos originais" (grupo que
filmou e realizou a contagem em seus próprios vídeos), "cientistas cidadãos
replicadores" (grupo de cientistas cidadãos que realizaram contagem da atividade
de voo em vídeos filmados por outros cientistas cidadãos) e especialistas
(pesquisadores que trabalham com abelhas e que realizaram as contagens em
vídeos filmados por cientistas cidadãos). Tais resultados permitiram a validação do
protocolo e indicaram alta qualidade dos dados produzidos por indivíduos que
participam de práticas científicas seguindo uma abordagem de ciência cidadã.

Referência: LEOCADIO, Jailson N. et al. Data Reliability in a Citizen Science

Protocol for Monitoring Stingless Bees Flight Activity. Insects, v. 12, n. 9, p. 766,
2021.
O foco do Capítulo V “Evaluation on a Bee focused Citizen Science Project:
Influences of participants profile in their learning”, apresentado no formato de
manuscrito de artigo científico Submetido ao periódico Sustainability, foi avaliar

20
aspectos do aprendizado dos cientistas cidadãos participantes do quarto
oferecimento do Curso de Extensão em Meliponicultura e Ciência Cidadã da
Universidade de São Paulo, relacionando os resultados de aprendizado obtido com
características do perfil dos cientistas cidadãos, considerando múltiplas dimensões.
A avaliação foi baseada em questionários pré e pós-curso. Todos os grupos
analisados mostraram um aumento significativo na aprendizagem em todas as
dimensões. Variáveis relacionadas ao nível de educação, área de conhecimento da
formação dos participantes e se o participante é meliponicultor ou não foram mais
influentes. Participantes sem prática de criação de abelhas apresentaram maior
aprendizado em algumas dimensões e, portanto, podem ser um bom grupo-alvo
para projetos de ciência cidadã focados em abelhas sem ferrão. Nossos resultados
indicam que mesmo quando pessoas com níveis mais baixos de educação formal
têm pontuações iniciais mais baixas, elas mostram um nível mais alto de
aprendizado. Iniciativas de ciência cidadã focadas em meliponicultura também
podem aumentar o conhecimento geral dos participantes e suas habilidades práticas
em relação às abelhas e à criação de abelhas sem ferrão. A avaliação da influência
do perfil dos voluntários pode contribuir para o planejamento de estratégias de
treinamento em outros projetos de ciência cidadã, o que pode, em última instância,
impactar positivamente a conservação das abelhas.
No Capítulo V é feita a principal contribuição acadêmica da presente tese,
trazendo novas perspectivas sobre a avaliação de cientistas cidadãos e a influência
de suas características de perfil no seu aprendizado e participção em projetos de
ciência cidadã.
Referência: Barbieri et al Evaluation of a Bee-focused Citizen Science
Project: Influences of participants’ profile on their learning. Sustainability. 2023.

Ao fim da tese, é feita uma breve discussão sobre as contribuições geradas

pelos trabalhos e uma conclusão com apontamentos de caminhos a serem
percorridos em próximos trabalhos decorrentes do projeto.

21
3 - CAPÍTULO I “Abelhas e Polinização”

DOI https://doi.org/10.4322/978-65-86819-20-5.s02c04.pt

Autores: Celso Barbiéria, Luis Flores-Pradob, Tiago Mauricio Francoya, Melisa G.

Geisac, Gerardo Pablo Gennarid, Mayara Faleiros Quevedoa

a
Escola de Artes, Ciências e Humanidades, Universidade de São Paulo, São Paulo, Brasil,
celso.barbieri@usp.br; tfrancoy@usp.br; mayara.faleiros@usp.br b Instituto de Entomología,
Universidad Metropolitana de Ciencias de la Educación, Santiago, Chile,
luis.flores@umce.cl
c
CONICET, INTA AER Cruz del Eje, Córdoba, Argentina, meligeisa@gmail.comd Instituto
Nacional de Tecnología Agropecuaria (INTA), Programa Nacional de Apicultura (PROAPI),
Famaillá (Tucumán), Argentina.
Abelhas na América Latina
Dentre as cerca de 20.000 espécies de abelhas existentes no mundo,
aproximadamente 5.000 são conhecidas para a América Latina, sendo distribuídas
em cinco famílias: Colletidae, Halictidae, Andrenidae, Megachilidae e Apidae. A
grande maioria apresenta hábito de vida solitário e entre 5 e 10% são sociais. As
abelhas pertencem à Ordem Hymenoptera e se caracterizam pela presença de uma
cintura estreita, que separa os dois primeiros segmentos do abdômen, sendo que o
primeiro está fundido com o tórax. O órgão usado pelas fêmeas para depositar os
ovos (ovipositor) é retrátil e, na maioria das espécies, encontra-se convertido em
ferrão para defesa.

Figura 1: Esquema da morfologia corporal de uma abelha

22
Abelhas sem ferrão
As abelhas nativas sem ferrão (Tribo Meliponini), são eussociais, altamente
organizadas e apresentam ferrão atrofiado, o que as impede de ferroar. São
pequenas (entre 3 e 10mm) e também podem ser distinguidas pela redução das
nervuras das asas.
Os meliponíneos são divididos em 33 gêneros e muito diversificados em cores,
hábitos e comportamentos. Nidificam em diferentes substratos, como ocos de
árvores, no solo, em muros e construções, aproveitando cavidades preexistentes.
São encontrados entre as latitudes 30ºS e 30ºN, sendo conhecidas, mundialmente,
aproximadamente 550 espécies, distribuídas em florestas e matas tropicais e
subtropicais na América, África, Ásia e Oceania. Em especial, a América Latina é
considerada como um possível centro de origem e especiação do grupo por
apresentar 75% da diversidade total, coexistindo as mais primitivas e as mais
derivadas espécies da tribo. Além disso, novas espécies são constantemente
descritas.

Figura 2: (A) Tetragonisca angustula e (B) Melipona quadrifasciata, duas espécies

de abelhas sem ferrão. Fotos: André Matos

Muitas espécies são culturalmente valiosas em diversas comunidades, devido às

suas contribuições alimentares, medicinais, simbólicas e materiais. Em particular, o
mel é o recurso mais utilizado pelas populações humanas desde os tempos antigos
e, a depender da espécie, a produção pode variar entre 250 ml a 5 L por colônia por
ano. Também é comum o aproveitamento de outras partes das colmeias, como
pólen, cerúmen, própolis e discos de cria. Sua criação racional é conhecida como
meliponicultura.

23
São importantes polinizadores da flora nativa e também têm sido muito utilizadas na
polinização de culturas a céu aberto e em estufas, uma vez que são muito
eficientes.

Abelhas das orquídeas

As abelhas das orquídeas (Tribo Euglossini) são importantes polinizadores de
diversas plantas com flores, especialmente de orquídeas, que são muito visitadas
pelos machos dessa tribo, em busca de fragrâncias, que são usadas na produção
de feromônios. Justamente por esse comportamento de busca por fragrâncias, os
machos dessa tribo são muito mais estudados do que as fêmeas, uma vez que são
atraídos por iscas de cheiro, o que facilita sua coleta.
As abelhas dessa tribo apresentam socialidade pouco desenvolvida, sendo que a
maioria é solitária ou comunal. Geralmente, podem ser encontradas nas partes altas
das árvores, no solo ou em cavidades pré-existentes. Possuem considerável
longevidade, especialmente as espécies de grande porte, que podem viver por
vários meses e são capazes de voar grandes distâncias.
Apresentam coloração metálica e uma língua longa, que, em algumas espécies,
pode exceder o dobro do tamanho do corpo, o que possibilita acesso a recursos
inacessíveis a outras abelhas e que potencializa a capacidade generalista desse
grupo. São dotadas de alta capacidade de dispersão, característica que, juntamente
com as anteriores, as torna capazes de sobreviver em ambientes perturbados. Esse
grupo, exclusivo da região Neotropical, distribui-se entre as áreas tropicais do
México até o norte da Argentina. Eventualmente são encontradas fora desses
limites, como no sul dos Estados Unidos devido a introduções acidentais. São
encontradas, principalmente, em florestas úmidas e regiões com baixas altitudes e,
onde podem constituir até 25% da diversidade de abelhas.
São conhecidas aproximadamente 220 espécies distribuídas em cinco gêneros,
sendo Euglossa, Eufriesea e Eulaema, compostos por espécies de vida livre, e
Exaerete e Aglae compostos por espécies cleptoparasitas.

24
 Figura 3: Euglossa sp. Foto: André Matos

Abelhas do gênero Bombus

As mamangavas de chão (Tribo Bombini) vivem em colônias que não são
perenes e se destacam por serem robustas e de grande porte (9-25mm).
Apresentam grande variedade de padrões de cor de sua pilosidade. As cores mais
comuns são o preto, amarelo, alaranjado e branco. Em nível mundial, são
conhecidos 39 subgêneros e 239 espécies. São conhecidos 14 subgêneros e 47
espécies na região Neotropical, 10 subgêneros e 25 espécies na América Central e
8 subgêneros e 21 espécies na América do Sul. Até o momento, já foram
documentadas introduções de três espécies na região Neotropical. São encontradas
em uma grande variedade de ambientes, desde o nível do mar até os 4400 metros
de altura, nos Andes.
Durante a fase ativa da colônia, os adultos forrageiam inclusive em condições
adversas, como períodos de chuva moderada e temperatura baixa. Também são
excelentes polinizadores em plantas que necessitam de polinização por vibração e
em flores de corolas compridas, o que torna estas abelhas alguns dos principais
polinizadores silvestres encontrados na América Latina. Dentre as espécies
conhecidas, Bombus pauloensis (sin. atratus) desperta um interesse particular, uma
vez que é distribuída amplamente na América do sul e apresenta uma série de
características interessantes para sua criação comercial em confinamento.

Abelhas solitárias
Na América do Sul são encontradas espécies solitárias que pertencem às
cinco famílias citadas anteriormente. A família Apidae possui a maior riqueza
específica a nível mundial e também na América Latina. A maioria das espécies

25
dessa família são solitárias, embora também sejam encontrados representantes de
todos os níveis de socialidade.
Alguns grupos taxonômicos se especializaram no uso dos recursos
necessários para o ninho. Por exemplo, na tribo Manueliini, endêmica do Chile e da
Argentina, as fêmeas constroem seus ninhos em galhos secos, caules ou troncos,
que consistem em células dispostas em série, uma ao lado da outra. Dentro de cada
uma, a fêmea forma uma massa (principalmente pólen e néctar), no qual deposita
um ovo, típico padrão de nidificação de espécies de abelhas carpinteiras, como as
tribos Ceratinini e Xylocopini. Algumas espécies da tribo Centridini se
especializaram em coletar óleo de certas espécies de plantas, com as quais cobrem
a superfície das células dos ninhos que as fêmeas constroem no solo, sendo este
mais um recurso floral explorado pelas abelhas. Entretanto, a maioria das espécies
solitárias constroem seus ninhos no solo. Tais ninhos consistem em um túnel que
desce mais ou menos reto, ou diagonalmente, de onde dutos secundários
geralmente bifurcam-se e terminam em câmaras com uma ou mais células, como
espécies das tribos Protandrenini, Diphaglossini e Halictini. Outras abelhas solitárias
nidificam em cavidades preexistentes e cortam folhas e pétalas para construir suas
células, como os gêneros Megachile e Anthidium.

Figura 4: (A) Halictidae em flor de manjericão e (B) Megachile sp. Fotos: André
Matos

Abelhas Apis mellifera

Apis mellifera (Linnaeus, 1758) é a abelha mais difundida do planeta, de
origem euro-asiática e que foi introduzida na América Latina no século XIX.
Atualmente, apresenta grande importância produtiva, alimentar e econômica. Já foi
declarada como "o ser vivo mais importante do planeta" em 2018, representando

26
toda a família Apidae, pela sua importância na polinização das culturas, pelas suas
contribuições na indústria alimentar e na saúde das pessoas, e pelo declínio global
de suas populações.
Caracterizam-se por formar colônias de até 80.000 indivíduos e constroem
favos verticais, onde as larvas são alimentadas continuamente até o último estágio
de desenvolvimento antes da formação da pupa e estágios pós-metamorfose.
São conhecidas pelo menos 31 subespécie, sendo que na América do Sul foram
introduzidas principalmente Apis mellifera ligustica, Apis mellifera carnica, Apis
mellifera mellifera, todas de origem europeia, e Apis mellifera scutellata, nativa da
África central e ocidental, introduzida no Brasil em 1956. Os híbridos das abelhas de
origem africana e europeia são chamados de abelhas africanizadas, que em menos
de 50 anos ocuparam grandes partes das Américas, não sendo encontradas
somente no Chile e na parte Central e Sul da Argentina, onde a apicultura é
praticada com subespécies de origem europeia.

Sugestão de leitura
1) Vit, P., Pedro, S. R., & Roubik, D. (Eds.). (2013). Pot-honey: a legacy of stingless
bees. Springer Science & Business Media.London.654pp.
2) Michener C. D. (2007). The bees of the world. The John Hopkins University Press,
Baltimore. 953pp.
3) Dressler, R. L. (1982). Biology of the orchid bees (Euglossini). Annual review of
ecology and systematics, 13(1), 373-394.
4)Nogueira Neto, P. (1997). Vida e criação de abelhas indígenas sem ferrão (No.
595.799 N778). Nogueirapis.
5) Root, A. I. (2005). El ABC y XYZ de la apicultura: enciclopedia de la cría científica
y práctica de las abejas (No. 638.1 ROOa 1984).

27
4 - CAPÍTULO II “A Buzz for Sustainability and Conservation: The Growing Potential
of Citizen Science Studies on Bees”

Sustainability 2021, 13(2), 959; https://doi.org/10.3390/su13020959

by: Sheina Koffler 1,*,†,Celso Barbiéri 2,†,Natalia P. Ghilardi-Lopes 3,Jailson N. Leocadio

4,Bruno Albertini 4,Tiago M. Francoy 2 andAntonio M. Saraiva 1,4

1 Instituto de Estudos Avançados, University of São Paulo, R. Praça do Relógio 109, São Paulo
05508-970, SP, Brazil; 2 Escola de Artes, Ciências e Humanidades, University of São Paulo, R.
Arlindo Bettio 1000, São Paulo 03828-000, SP, Brazil; 3 Centro de Ciências Naturais e Humanas,
Federal University of ABC, R. Arcturus 3, São Bernardo Do Campo 09606-070, SP, Brazil; 4 Escola
Politécnica, University of São Paulo, Avenida Professor Luciano Gualberto 158, Tv. 3, São Paulo
05508-010, SP, Brazil *Author to whom correspondence should be addressed.† Shared co-first
authorship.

Abstract

Expanding involvement of the public in citizen science projects can benefit both
volunteers and professional scientists alike. Recently, citizen science has come into
focus as an important data source for reporting and monitoring United Nations
Sustainable Development Goals (SDGs). Since bees play an essential role in the
pollination ecosystem service, citizen science projects involving them have a high
potential for attaining SDGs. By performing a systematic review of citizen science
studies on bees, we assessed how these studies could contribute towards SDG
reporting and monitoring, and also verified compliance with citizen science
principles. Eighty-eight studies published from 1992 to 2020 were collected. SDG 15
(Life on Land) and SDG 17 (Partnerships) were the most outstanding, potentially
contributing to targets related to biodiversity protection, restoration, and sustainable
use, capacity building and establishing multi-stakeholder partnerships. SDG 2 (Zero
Hunger), SDG 4 (Quality Education), and SDG 11 (Sustainable Cities and
Communities) were also addressed. Studies were found to produce new knowledge,
apply methods to improve data quality and invest in open-access publishing.
Notably, volunteer participation was mainly restricted to data collection. Further

28
challenges include extending these initiatives to developing countries, where only a
few citizen science projects are underway.
Keywords: bee monitoring; beekeeping; citizen science principles; pollination; sustainable
development goals

Graphical Abstract

1. Introduction

Citizen science (CS), which can be defined as the involvement of (usually

unpaid) volunteers in the scientific process (e.g., data collection, analysis, and
interpretation) [1,2], has been used for different purposes, including biodiversity and
environmental monitoring of both terrestrial and aquatic ecosystems. Data produced
through citizen science initiatives is speeding up, in an unprecedented way,
understanding of patterns and functions in biodiversity [3,4,5], thereby contributing
towards natural resource management, environmental protection, and policymaking,
as well as fostering public input and engagement [6,7,8]. Increasingly, the knowledge
that is deeply integrated across disciplines and co-produced with non-academic
stakeholders [9] is needed to achieve the 17 United Nations Sustainable
Development Goals (SDGs), which aim for a better and more sustainable future for
all by 2030, seeking the end of poverty, the improvement of health and education,
the reduction of inequalities, the stimulus to economic growth, while tackling climate

29
change and working to conserve our oceans and forests [10]. More recently, citizen
science has been recognized as a source of data for SDG reporting and monitoring,
thus potentially contributing to 76 of the 244 SDG indicators [11]. Certain features of
CS data are of extreme relevance, such as: spatial reference, resolution and extent;
duration and temporal resolution; thematic subject areas, definitions and resolution;
data purposes, use, collection, processing and management (if data is findable,
accessible, interoperable and reusable); and levels of participant involvement [12]. If
used in accordance with ethical and scientific principles [13,14], CS has enormous
potential to expand knowledge about global biodiversity, reducing taxonomic and
spatial biases in global biodiversity data sets, moving beyond data on the occurrence
of single species and providing further understanding of ecological interactions
among species or habitats [15].

Since they are generally small-sized and can easily fit into photographs, insects
pose significant opportunities for citizen science approaches, more so than with most
other biological groups. Nonetheless, apart from the possibility of sampling in many
different situations, there still remain several pertinent, and as yet, unanswered
scientific queries [16]. Bees are excellent subjects for mutually integrating citizen
science projects and SDGs, since they comprise the most dominant pollinating taxon
[17] and their ecological importance is widely recognized by the public [18]. Although
bees are not the most diverse group of pollinators (as butterflies and moths, beetles,
and flies show higher species richness, [17] and other pollinating insects contribute
significantly to flower visitation and fruit set in crops globally [19]), bees are still the
major pollinating group of wild and crop plants [20]. The global economic value of
pollination has been estimated as between US$ 235 and 577 billion [20]. However,
the diversity of wild and managed bees has crucial ecological, economic and social
importance beyond crop pollination [21]. Indeed, bees were recognized as
contributing to 15 of the 17 SDGs and at least 30 relevant SDG targets. They can
easily be linked to SDGs, such as SDG 15 (Life on land), SDG 11 (Sustainable cities
and communities), and SDG 12 (Responsible consumption and production) [11].
They can also be successfully used in educational programs, especially those
dealing with the environment, ecology and conservation [22], directly linked to SDG 4
(Quality Education).

30
 Here we performed a systematic review of studies combining citizen science and
bees. In each case, the assessment focused on addressing each study to the
appropriate SDG, besides evaluating system traits, citizen participation, methods
employed, and research questions. Our aim was to understand the potential
contribution of these particular citizen science studies to SDG reporting and
monitoring, and how they conformed to ECSA principles, in order to place in
evidence opportunities for enhancing practices in this field.

2. Material and Methods

The review carried out in this study followed the guidelines proposed by PRISMA
[23], which defines a systematic review as a study employing appropriate and explicit
methods to identify, select and critically evaluate relevant research through data
collection and analysis.

2.1. Search Process

The survey of the literature was carried out in two steps: a naive search with
pre-selected author terms, followed by a second step with less biased terms after
using the litsearchR package [24]. All analyses were performed using the R version
4.0.0 (R Core Team 2020). litsearchR, besides automating several steps of
systematic review, employs the Rapid Automatic Keyword Extraction (RAKE)
algorithm to identify potential keywords initially omitted by researchers, thus
improving reproducibility and reducing bias.

The searches in the literature were performed in Web of Science and Scopus
databases, which returned 102 and 114 articles, respectively, during ‘naive search’.
The initial selection of the search terms was according to the PECO (Population,
Exposure, Comparator, Outcome) framework [25]. Population was represented by
“bees”, Exposure by “citizen science”, and Outcome by research aims and questions
which were our interest in this study. Citizen science terms included those described
by Eitzel et al. [26], which provide a historical overview of terminology in CS. No
Comparator terms were used. Sequentially, the litsearchR algorithm package was
applied to this initial list of articles, according to the protocol and parameters

31
suggested by the package authors [27]. After identifying new synonyms and related
words for bees and citizen science, research was resumed on December 7 to
include novel recently published articles. The final results were 122 articles in Web of
Science and 134 in Scopus (total n = 256). The queries searched the terms in the
title, abstract and keywords and considered all records available in the databases
(the terms in italics were suggested by the litsearchR package):

(( “bee” OR “bees” OR “Apoidea” OR “Antophila” OR “honey bee” OR “honeybee” OR

“apis mellifera” OR “beekeep *” OR “bee colon *” OR “coloss” OR

“queen problem *” OR “young queens” OR “bumble bee *” OR “bumblebee”

“brood cell*” OR “brood comb” OR “native stingless” OR “australian stingless” OR

“frieseomelitta ningra” OR “geotrigona acapulconis” OR “lestrimelitta chamelensis” OR

“melipona fasciata” OR ”scaptotrigona hellwegeri” OR “pot-honey” OR

“african carder”)

AND

(‘citizen science” OR “participatory science” OR

“crowd * science” OR “volunteer * monitoring” OR “networked science” OR

“collaborative research” OR “collaborative monitoring” OR

“collaborative science” OR “participatory action research” OR “community action research”

OR “crowdsourcing” OR “community-based participatory research” OR “local knowledge” OR

“volunteered geographic information” OR “public participation in scientific research” OR

“community science” OR “citizen scientist”))

32
 Of these, 101 were removed through duplicate analysis, six were introductions of
conference proceedings (and were also removed), and 149 remained as potentially
relevant study items. In addition, 13 others were selected through citation in the
revised studies or author personal knowledge, resulting in a total of 162 studies. Our
search strategy was as inclusive as possible, in an effort to include studies that
fulfilled the established requirements, even when the term “citizen science” was not
present. This is especially important since an assessment of ornithological studies
showed that many studies employing CS data did not explicitly mention volunteer
participation in data collection [28], which may also have interfered in our search
results.

2.2. Collected Data

Titles and abstracts were read to identify studies that did not conform to the
requirements for analysis, e.g., could not be characterized as CS, did not include
bees (only other pollinators), or which only mentioned or recommended CS, but did
not include citizen scientists in any step of procedures. Following Eitzel et al. [26]
recommendations, we only considered as citizen science those studies in which
volunteers were actively involved in some aspect of the project and were informed
how their data was going to be used. Thus, data gathered at online databases
(without the owners’ knowledge or consent) and, thus not related to CS programs,
were not considered. In addition, studies regarding local knowledge assessments
were only included if participants were knowledge producers (not study subjects,
e.g., Smith et al. [29]). After screening, 74 articles were removed and 88 remained
for review (Supplementary Material 1). A subset of 40 articles were read, each by
two researchers, to validate and standardize the terms and categories used for each
indicator up for analysis (Table 1). The remainder were distributed equally between
them. All articles were analyzed, and the most prominent UN SDG related to each
one was defined. SDG 2 (Zero Hunger) was related to studies of the influence of
bees on agricultural production, SDG 4 (Quality Education) to those in which bees
were used to promote scientific education (this process was the main focus of the
study), SDG 11 (Sustainable Cities and Communities) to studies of the biology of
bees on urban landscapes, SDG 15 (Life on Land) to those aimed at investigating
biological or ecological aspects of bees in general, such as species identification,

33
occurrence and distribution, and their interaction with plants, among others, and
SDG 17 (Partnerships) to studies in which establishing a partnership, such as
recruitment, engagement, retention strategies or co-creation of platforms, or
validating a citizen science project, the case of pilot-testing of protocols, and data
quality analysis, were central to research. SDGs are related to distinct targets, each
measured by distinct indicators. Hence, we also indicated which targets could be
addressed and their indicator tier classification [30]. Indicators are classified into
three tiers, tiers I and II having established methods and standards. However, for tier
II indicators data collection is not regular countrywide. Currently, the global indicator
framework does not include any tier III indicators, which are those with no methods
and standards available yet.

Table 1. Variables assessed in each study retrieved in the systematic review. Details are given for the
variable name, information source (A: article or P: project), definition and levels considered for each
variable, and which ECSA principle was being assessed (see also Table S2).

Group Variable name Source Definition ECSA

principle
Study Proponents (adapted A Affiliation of study authors -
information from CUNHA 2017)
Funding source A Funding institution cited/acknowledged in -
(adapted the article
from CUNHA 2017)
Reach A Spatial scale (local, regional, or global) -
Country A Country or countries where the project was -
performed
Project duration P Project length and status (short or -
long-term, active or finished)
SDG A UN Sustainable Development Goal -
addressed by the study (see explanation in
the text)
Research subject A Research areas explored (when more than 9
one aim was declared, only the main results
were considered)
Hypothesis-led A Whether authors clearly state that there is a 2
study question or hypothesis to test
Data quality (modified A Strategies employed to improve data quality 3, 6
from Wiggins 2011) in the citizen science projects

34
Project Project name P Citizen science project name -
information
Project purpose P Aim of volunteer participation in the project -
(modified (biological recording, biological monitoring,
from Pocock 2017) crowd-sourcing, or creating technology
platforms)
Degree of A Whether study was contributory, 1, 3, 4
participation collaborative, co-created, or used
(modified crowd-sourced data.
from bonney2009a)
Studied system Animal group A Which animal groups were studied -
Taxon A Taxonomic name of the focused group -
Sociality A Sociality level of the bees studied -
Participant Number of A Number of participants contributing to the -
information participants project. Descriptive statistics were based on
exact numbers provided by the study
(approximations were not considered)
Volunteer profile A Profile of participants. “General -
public†was inferred when no other profile
was mentioned
Recruitment A Methods employed for recruiting -
participants
Communication A Methods and tools used to train participants 3
and deliver relevant information about the
project
Volunteer A Learning outcomes and perceptions of 3
assessment (what?) volunteers
(modified
from kelemen2018)
Volunteer A Mechanisms for volunteer assessment 3
assessment (how?)
Volunteer A At which step volunteers were assessed 3
assessment (when?) (pre/post survey)
Ethics Open access A Publication type regarding accessibility or 7
commitment whether the study was a conference paper
Feedback to A Whether authors mention if any feedback 5
participants was given to the volunteers
Acknowledgements A Whether citizen scientists were 8
acknowledged in the study.

35
 Additionally, the compliance with ECSA principles [13] was analyzed for each
article, as follows: 1. CS projects actively involve citizens in a scientific endeavor that
generates new knowledge and understanding; 2. CS projects should have a genuine
scientific outcome. 3. Both professional scientists and citizen scientists benefit from
taking part; 4. Citizen scientists may, if they so wish, participate in multiple stages of
the scientific process; 5. Citizen scientists receive feedback from the project; 6. CS is
considered a research approach like any other, with limitations and biases that
should be considered and controlled; 7. CS project data and metadata are made
publicly available and where possible, results are published in an open-access
format; 8. Citizen scientists are acknowledged in project results and publications; 9.
CS programs are evaluated for their scientific output, data validity, participant
experience, and wider societal and policy impact; and finally, 10. The leaders of CS
projects should take into consideration the legal and ethical issues surrounding
copyright, intellectual property, data-sharing agreements, confidentiality, and
attribution, as well as the environmental impact of any activity (see Table 1 for the
correspondence of each variable and ECSA principle and Table S2 for detailed
criteria used for each principle). Even though distinct aspects are considered in each
individual principle, compliance with the principle was treated as a binary variable,
which means that when more than one indicator was proposed, we considered the
principle as fulfilled if at least one of them was contemplated in the study. Not all
aspects covered by the ECSA Principles could be assessed in our analysis because
they were not explicitly mentioned in the published results. We highlight here the
difficulty to evaluate all aspects of the 9th ECSA principle, because of the intrinsic
complexity for the measure of project outputs and impact [14]. The 10th principle
was not evaluated because a deep analysis of each project would be necessary
since the information presented in the manuscripts does not always include the legal
and ethical aspects of the CS initiatives in detail. Despite these limitations, we
believe our analyses provide an interesting framework for systematically assessing
CS research, which may be further expanded in future studies.

36
3 Results
Of all the 88 scientific studies that were collected, 81 were peer-reviewed articles
and seven conference papers. From 1992 to the present, there has been a constant
increase in the number of studies published per year, reaching a maximum of 19 in
2019 (Fig. S1). Most studies were undertaken on a local scale (Fig. 1) and
comprised data from long-term citizen science projects (73%, n = 64), including 47
ongoing projects.

Figure 1. Global distribution of citizen science studies of bees. All the reported countries were
gathered for each study. The study was not represented when individual countries were not reported
in the article ( e.g., “Europe”). The map was constructed using the ‘rworldmap’ R package [35].

37
3.1. Sustainable Development Goals

The studies were mainly addressed to SDG 15 (Life on Land, 52.3%), followed
by SDG 17 (Partnerships, 29.5%), with the remainder to SDGs 2, 4, and 11 (Zero
Hunger, Quality Education, and Sustainable Cities and Communities,
respectively—Figure 2a). Even though the reviewed studies were not explicitly
related to any SDG, data, and findings produced could be used to monitor and
implement 12 tier I and 9 tier II indicators (Table S3). The main research subjects
were beekeeping, distributional ecology (with 6 studies focusing on invasive
species), data quality, natural history, plant-pollinator interactions, volunteer
assessment, and landscape ecology (Figure 2b). Few studies focused on population
ecology, agricultural practices, toxicology, or were descriptions of new projects.
Project purposes were biological recording (63.9%), biological monitoring (30.7%),
providing technology platforms (2.3%), and crowd-sourcing (1.1%).

Figure 2. (a) Absolute number of studies related to each SDG identified in the review process and (b)
Proportion of research subjects expressed in the studies analyzed (number of studies is represented
after the bar).

38
 Bees were the only study system in 76% of the studies, whereas 24% included
other animals, such as insects, invertebrates, or birds (complete dataset available in
Supplementary Material 1). Among those, 65.9% focused on social bees, 21.6% on
both social and solitary species, and 12.5% only on solitary species. Most studies
gathered data for Bombus spp. (35.2% with a focus on the genus level or the specific
species), and honey bees (Apis mellifera, 26.1%. Stingless bees were only
investigated in three studies. While honey bee studies were frequently related to
beekeeping (83%), bumble bee studies showed a more diverse pattern of research
subjects. Solitary bee studies, on the other hand, were usually related to
distributional ecology. Several studies (18%) were based on data provided by
long-term citizen science projects on these bee groups, such as Bee Watch, Bumble
Bee Watch, and COLOSS (COLOSS survey of honey bee colony losses).

Funding was mainly provided by governments (68.2%), followed by

non-governmental organizations (NGOs, 28.4%), universities (22.7%), and the
private sector (13.6%). While two studies declared no funding to report or personal
funding, ten made no mention (Figure S2a). University members were authors in
93.2% of the studies, while governments and NGOs were represented in 39.8% and
38.6%, respectively. Three studies also had authors from the private sector and one
from a school (Figure S2b). The number of participants in each project varied from 2
to 28,629 (Figure S3); however 32% of the studies do not mention how many
participants were involved. Although in several cases (36.4%) there was no mention
of how participants were recruited, the most frequent manner was through digital
media (38.6%), or related organizations (43.2%), such as beekeeper or gardener
associations. Participant communication and training were either online (51.8%),
through presencial meetings and workshops (22.9%), or through manuals (24.1%).

39
3.2. ECSA Principles

The reviewed papers scored points on a scale from 2 to 9 (Figure 3). All the
reviewed papers scored points for principles 1 and 2. We considered these principles
classificatory to be included in the analysis. The average score of the 88 reviewed
papers is 6.7, the minimum 3 (1.1%) and the maximum 9 (7.9%). A total of 38 papers
scored 7 points (43.2%). Excluding principles 1 and 2, the most frequent principles
were Principle 4 (97.7%) and Principle 6 (94.3%), while the less frequent ones were
Principle 9 (35.2%) and Principle 5 (23.9%).

Figure 3. Compliance with ECSA principles (a) Number of studies according to total score (number of
principles fulfilled) and (b) Proportion of studies following each ECSA principle.

When considering the first and second ECSA principles, notably hypotheses or
scientific questions were explicitly stated in 50% of the studies. Although several
(48.8%) relied on volunteers from the general public, without targeting any specific
group, beekeepers, bee enthusiasts, and students were target groups in some.
Considering volunteer participation (fourth ECSA principle), data collection was the
main task performed by citizen scientists, and 93.2% of the studies were classified
as contributory. Two studies were collaborative, three co-created, and one relied on
crowd-sourcing. From all selected studies, 37.5% contained information on feedback
to participants (fifth ECSA principle), and 53.4% were published as open access
(seventh ECSA principle). Volunteers were acknowledged in 79.5% of the studies
(eighth ECSA principle), and two articles included participants as authors [29,36].

40
 Few studies (22%) included some kind of volunteer assessment (ninth ECSA
principle) and those that did, focused on interest, motivation, behavior, knowledge,
and perception. Assessment, applied through questionnaires, was generally applied
after volunteer participation (Table S1). As to data quality evaluation (ninth ECSA
principle), different strategies were employed, with a maximum of five per study
(Figure 4). Digital vouchers (photographs submitted by citizen scientists), expert
review of data, use of structured protocols, and training of participants were the most
frequent strategies to improve data quality. Volunteer personal knowledge, usually
related to beekeepers’ experience in analyzing hive conditions, was also exploited to
improve data quality.

Figure 4. Proportion of data quality control strategies employed in citizen science studies of bees.
Each study could apply more than one strategy. Number of studies is represented after the bar.

41
4. Discussion

Our results revealed that five of the 17 SDGs could be addressed by citizen
science studies of bees. In general, the aims were to expand basic bee knowledge,
investigate applied aspects of beekeeping and conservation, and explore the
potential of CS as a research and educational process. The recent increase noted in
the number of published works follows a more general trend in scientific publications
in this field and indicates the acceptance of citizen science by the research
community towards mainstreaming this research avenue [37]. Publishing results
from citizen science projects in peer-reviewed journals and conferences also fulfills
the first and second citizen science principles, as citizen participation is generating
new knowledge and understanding, while at the same time producing genuine
scientific outcomes [13]. In addition, the growing number of studies is contributing to
the generation of large-scale data that can be employed as a non-traditional data
source for SDG monitoring and implementation [12]. In a previous analysis, bees
were related to 30 targets across SDGs 1 and 15, demonstrating a great potential to
contribute to SDGs achievement [21]. However, SDGs 16 and 17 were not taken into
consideration by Patel et al. [21], as these goals focus on governance and policy,
which nevertheless can be targeted through CS. Another positive factor is the
increasing recognition by the public that bees are important providers of pollination
services [38], a possible motivation for recruiting volunteers for participation in citizen
science projects. In fact, the main motivational factors declared in a survey with
citizen scientists from a bee monitoring program were contributing to scientific data
collection and aiding in bee conservation [39]. Since bees are the most dominant
pollinators of flowering plants [17] and bee diversity contributes to increasing crop
production [40], studies combining agriculture and bees were related to SDG 2 (Zero
Hunger). Specifically, CS projects may contribute to ensuring sustainable agriculture
and increasing productivity (target 2.4). In this context, as bee monitoring in crop
fields is a potential tool for sustainable agriculture, this could be implemented in
collaboration with agriculture stakeholders [41]. For instance, the citizen science
studies reviewed here focused on assessing data quality of bee monitoring protocols
in crop fields [42], as well as management strategies for assuring adequate
pollination services for food production [43].

42
 SDG 4 (Quality Education) is related to articles regarding CS projects for
schools. Studies thus addressed would contribute to relevant and effective learning
outcomes, education for sustainable development, and teacher qualification (targets
4.1, 4.7, and 4.c, respectively). Even though SDG targets and indicators focus on
literacy and numeracy in primary and secondary education, we highlight the
importance of scientific literacy for sustainability comprehension and implementation,
which can be positively impacted by participation in citizen science projects [44,45].
Regarding vocational and technical skills, Patel et al. [21] also suggest that training
for beekeeping may provide equitable opportunities for men, women, and indigenous
people possessing traditional knowledge, which can be considered as an innovative
opportunity for citizen science projects in SDG 4. SDG 11 (Sustainable Cities and
Communities) was explored when investigating bee response to urbanization, a key
conservation factor when considering the growing urban sprawl. Reviewed studies
regarding this SDG mainly focused on landscape ecology, hence related to
protecting and safeguarding the world’s natural heritage and providing access to
green and public spaces (targets 11.4 and 11.7). For instance, CS data revealed
foraging resource availability was related to solitary bee nesting [46], and how the
proportion of impervious surface affected bee communities [47]. Thus, there is a
positive relationship between bee presence and gardens, urban green spaces and
the remaining natural ecosystems within cities, which in turn, benefit from bee
pollination. None of the studies focused on employing bees for air quality monitoring
in cities, another potential application for citizen science and SDG 11 [21]. Expanding
knowledge of basic bee biology and ecology contributes to SDG 15 (Life on Land),
potentially promoting conservation and the sustainable use of terrestrial ecosystems.
CS projects provide large-scale spatial and temporal data, allowing for estimating
species distribution and assessing extinction risk [48,49,50]. These results may
ultimately be useful for identifying potential areas for biodiversity protection and
informing national conservation policies, hence attaining targets 15.1, 15.4, 15.5, and
15.9. Regarding alien species (target 15.8), the high sampling effort in citizen
science projects could facilitate recording and monitoring invasive processes [51,52].
Furthermore, as bees are important pollinators [17], data obtained by citizen
scientists on flower visitation and pollination outcome [53,54], could ultimately
contribute to conservation and restoration programs that rely on effective pollination
(targets 15.1–15.5). Despite the increasing number of CS studies on bees, there is a

43
bias for social species, especially honey bees and bumblebees. Solitary bees, which
comprise the majority in bee diversity [55], are largely unknown by the public [38],
contrary to what occurs with managed bees. Thus, gaps in bee diversity are still
found in CS research, with few projects focusing on native solitary and stingless
bees, which exhibit high species richness resulting in difficulties in species
identification by non-experts. This is especially important in tropical countries, where
species diversity is high. All told, partnerships with beekeepers [56] could provide an
opportunity for the sustainable use of biodiversity, as well as fair and equitable
sharing of benefits from the use of genetic resources (targets 15.1 and 15.6).
Establishing partnerships (SDG 17) is essential for successful CS projects. Inducing
capacity building and multi-stakeholder partnerships, and involving the various social
sectors that are central elements in citizen science projects, are in line with targets
17.9, 17.16, and 17.17. In this respect, our results corroborate the findings of Cunha
et al. [31], showing that governments and NGOs play major roles in building and
funding partnerships, whereas in the private sector this is less so. Private sector
participation (17.17) could lead to inducing companies to support science and
sustainability, hence rewarded with innovations in technology and production (SDG 9
Industry Innovation and Infrastructure). On the other hand, as study proponents are
mainly researchers, this could affect the intended role of each stakeholder. Indeed,
the level of participation in most of the studies reviewed was contributory, thereby
indicating that citizen scientist participation is restricted to collecting or processing
data [33]. According to the fourth principle of citizen science, citizen scientists may, if
they so wish, participate in several stages of the research process [13]. Similar to
previous assessments, contributory approaches were more frequent [2]. Even
though requiring more effort and engagement from both researchers and citizen
scientists, involving higher levels of citizen participation in collaborative and
co-created projects is also relevant as they promote greater ownership and may
bring contributions driven by the needs of the community that may be related to key
SDG targets [12]. The scarcity of co-created projects may be explained once one of
the main disadvantages in CS approaches is related to the amount of effort and
difficulty to execute a protocol or participate in the project. The co-created and
collaborative projects usually need more effort from the professional scientists to
maintain the volunteers committed and manage the research, because the data
acquisition is not under the professional team direct control [57]. According to

44
Martens [58], spatial scale is an essential factor in sustainable development. Most of
the reviewed studies concentrated on a local scale, which contributes to monitoring
volunteers, their activities, and expectations [57]. In addition, citizen science projects
contributing to monitoring at the local scale have greater potential to implement
SDGs in specific contexts, and if successful, feasible for scaling up when attempting
to reach a global level [12]. Scaling CS initiatives to the global level may contribute to
bee monitoring programs, which can provide essential information on how pollinators
face global change [59]. Indeed, participatory research has been increasingly
indicated as a powerful strategy for long-term pollinator monitoring, suggesting an
avenue for mainstreaming CS in bee and pollination research and advocating for
funding those initiatives [60,61,62]. Since the Global Biodiversity Information Facility
(GBIF) already accepts CS data, including these data on research will become more
common and highlights the importance of data sharing practices. Currently and
according to previous studies, most projects were carried out in developed countries,
mostly in Europe and North America [5,63]. The United Kingdom stands out as a
leader in citizen science projects involving bees, which was to be expected, due to
the UK’s centenary tradition in biodiversity monitoring projects [64]. On the contrary,
in developing countries, where access to research funding is more restricted and
levels of formal education lower, there are less citizen science projects and added
difficulties in volunteer engagement [63,65]. Investing in citizen science projects on
developing countries as a strategy to achieve SDGs is highly relevant because these
countries concentrate the greatest and most important biodiversity hotspots, are
responsible for the maintenance of numerous ecosystem services, and show high
levels of poverty and inequality [66]. Citizen science thus emerges as a promising
way to engage local communities in conservation projects, besides being a
potentially cheaper way to monitor biodiversity, especially important in developing
countries. Furthermore, citizen science initiatives focused on beekeeping should be
supported in developing countries, because this activity can be a promoter of
sustainability in its social, economic, cultural, and ecological domains [22], being
valuable to reduce social and gender inequalities [21]. Regarding best practices and
the use of correct terminology, during our search, several studies using the term
“citizen science” were found. However, they were not considered as CS in our study,
according to current definitions and so were excluded from our analyses [26]. Even
though the term “citizen science” was found in keywords of some studies, authors

45
only mentioned or suggested CS. Two allegedly CS studies used data collected from
people stored on social media platforms, not involving citizen scientists on the
science-making process, and being in conflict with the ECSA principles. The
distinction between this kind of data mining and the crowdsourced level of
participation in citizen science must be highlighted because consent is an ethical
principle of this research field. The term citizen science should be carefully used in
scientific publications, once this field is a valid scientific approach like any other with
their own strengths and limitations and the appropriate use of terminology
contributes to its recognition as a field of study [26]. Volunteer assessment was
rarely explored in the studies, thereby hampering an evaluation of outcomes from the
volunteer’s point of view (third principle, [13]). Knowledge of the volunteer’s
motivations is crucial for keeping them engaged, especially in the case of long-term
citizen science projects [57]. However, we highlight that the analyzed citizen science
projects may present volunteer assessment protocols, even though this feature was
not exploited in the publications. Results from citizen scientist outcomes are highly
valuable when designing new or improving long-term projects [67]. Recruitment and
communication strategies employed in citizen science projects also exert great
influence when engaging citizen scientists [57]. Recruiting a particular profile
volunteer with a close relationship with the research subject may be a simple manner
of improving data quality since the volunteer can rely on personal knowledge to run
the protocols [32]. This strategy was used in several of the studies surveyed here,
where beekeepers were recruited in large numbers through already existing social
organizations [54,56]. Volunteer participation was also assessed as to the quality of
data, always a major concern in citizen science studies [12,32]. In most of the cases,
the focus was on reliability in species identification, still a challenge in citizen science
projects on biodiversity. Strategies to overcome taxonomic uncertainty may involve
data validation by experts, identification restricted to higher-level groups or
non-natural groups and focusing on easily identifiable species [68,69].

46
 Feedback to volunteers on the research is of utmost importance for maintaining
citizen scientist motivation and collaboration throughout the project [70]. More so,
communicating project outcomes represents an ethical principle for professional
researchers (fifth and tenth principles [13]). Furthermore, high-quality science
communication is essential not only for spreading specific knowledge, but also
building up trust between the population and scientific community [71]. Although
there was mention of the various communication strategies, several did not mention
feedback to participants, possibly since this feature was unrelated to the research
aim. Nonetheless, numerous studies were published as open access, and most
acknowledged volunteers, thus in accordance to the seventh and eighth principles of
citizen science [13]. Even though open-access publishing in citizen science is still
hampered by elevated costs [72], citizen science articles on bees were more
frequently published in this format (56%), in comparison with statistics for general
publication (20.4% [73]). Notwithstanding, volunteer acknowledgment should be
strongly recommended, seeing that in 22% of the studies there was none.

5. Conclusions

Since sustainability can only be achieved through an intergenerational approach

[58,74], establishing firm partnerships is an essential step, whereby citizen science
can act as a powerful strategy for providing data and implementing SDGs [12]. Our
findings revealed that existing citizen science projects are already contributing to
scientific research, and even though none were directly aligned to SDGs, studies
implicitly related to and data derived from these projects are linked to 21 indicators
from 18 SDG targets. Among these, nine are tier II indicators without regular data
production, and thus can be positively impacted by citizen science projects
contributing with data [12].

47
 Nevertheless, citizen science research on bees still presents major gaps, such
as the lack of volunteer assessment, which would significantly contribute to building
efficient volunteer engagement strategies, improving learning outcomes, and
promoting meaningful experiences. To include citizen science in the SDG workflow
and implementation in local contexts, these issues should be dealt with. The
inclusion of citizen scientists in all the steps of the scientific process is still
uncommon, and should be fostered in further studies. Major challenges are citizen
science in developing countries, where investments on research are constrained and
budget cuts frequent [75,76]. Few citizen science papers have been produced in
developing countries, especially in the southern hemisphere. Thus, these represent
priority areas for formulating participatory and co-created projects aiming at
achieving several SDGs related to bees, beekeeping, and biodiversity.

Supplementary Materials

The following are available online at

https://www.mdpi.com/2071-1050/13/2/959/s1, Supplementary Material 1:
Complete dataset on citizen science studies assessed in this systematic review;
Supplementary Material 2: Figure S1: Publication of citizen science studies on bees
from 1992 to 2020, Figure S2: Funding source and proponent affiliation of the citizen
science studies on bees, Figure S3: Number of participants engaged in the citizen
science studies on bees, Table S1: Variables assessed in the systematic review and
summarized results, Table S2: Compliance criteria for the ECSA Principles, Table
S3: SDG targets and indicators related to the citizen science studies on bees.

48
Author Contributions

Conceptualization, S.K. and C.B. methodology, S.K., C.B., N.P.G.-L., J.N.L., B.A. and
T.M.F.; validation, S.K., C.B., N.P.G.-L., J.N.L., B.A. and T.M.F.; formal analysis, S.K.
and C.B.; investigation, S.K. and C.B.; data curation, S.K. and C.B.; writing—original
draft preparation, S.K., C.B., N.P.G.-L., J.N.L. and T.M.F.; writing—review and
editing, S.K., C.B., N.P.G.-L., J.N.L., T.M.F. and B.A.; visualization, S.K., C.B., J.N.L.
and B.A.; supervision, N.P.G.-L., B.A., T.M.F. and A.M.S.; project administration,
A.M.S., T.M.F., N.P.G.-L. and B.A.; funding acquisition, A.M.S. All authors have read
and agreed to the published version of the manuscript.

Funding

This research was funded by Fundação de Amparo e Apoio à Pesquisa do

Estado de São Paulo (FAPESP, grant numbers 2018/14994-1 and 2019/26760-8).
This study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal
de Nível Superior—Brazil (CAPES)—Finance Code 001; C.B. grant number
88882.377160/2019-01; J.L grant number 88882.333367/2019-01, and by Conselho
Nacional de Desenvolvimento Científico e Tecnológico—Brazil (CNPq), A.S. grant
number 312605/2018-8.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

49
Data Availability Statement

Complete datasets are provided as Supplementary Material.

Acknowledgments

We thank all citizen scientists and researchers from the studies reviewed here for
their contribution to bee conservation and sustainability. We also would like to
acknowledge researchers from the SURPASS2 project (Safeguarding pollination
services in a changing world) for their support to our study. The SURPASS2 project
is funded under the Newton Fund Latin America Biodiversity Programme:
Biodiversity—Ecosystem services for sustainable development, awarded by the
UKRI Natural Environment Research Council (NERC: NE/S011870/2), in partnership
with the Argentina National Scientific and Technical Research Council (CONICET
1984/19), Brazil/São Paulo Research Foundation (FAPESP 2018/14994-1), and
Chile National Agency for Research and Development (ANID NE/S011870/1).

Conflicts of Interest

The authors declare no conflict of interest.

Abbreviations
The following abbreviations are used in this manuscript:

SDG Sustainable Development Goals

CS Citizen Science
ECSA European Citizen Science Association
PRISMA Preferred Reporting Items for Systematic Reviews and Meta-Analyses
RAKE Rapid Automatic Keyword Extraction
PECO Population, Exposure, Comparator, Outcome
NGO Non-Governmental Organizations
GBIF Global Biodiversity Information Facility

50
References

1. Bonney, R.; Cooper, C.B.; Dickinson, J.; Kelling, S.; Phillips, T.; Rosenberg,
K.V.; Shirk, J. Citizen Science: A Developing Tool for Expanding Science
Knowledge and Scientific Literacy. BioScience 2009, 59, 977–984. [Google
Scholar] [CrossRef]

2. Pocock, M.J.O.; Tweddle, J.C.; Savage, J.; Robinson, L.D.; Roy, H.E. The
diversity and evolution of ecological and environmental citizen science. PLoS
ONE 2017, 12, e172579. [Google Scholar] [CrossRef]

3. Schmeller, D.S.; Henry, P.Y.; Julliard, R.; Gruber, B.; Clobert, J.; Dziock, F.;
Lengyel, S.; Nowicki, P.; Deri, E.; Budrys, E.; et al. Advantages of
volunteer-based biodiversity monitoring in Europe. Conserv. Biol. 2009, 23,
307–316. [Google Scholar] [CrossRef]

4. Theobald, E.; Ettinger, A.; Burgess, H.; DeBey, L.; Schmidt, N.; Froehlich, H.;
Wagner, C.; HilleRisLambers, J.; Tewksbury, J.; Harsch, M.; et al. Global
change and local solutions: Tapping the unrealized potential of citizen science
for biodiversity research. Biol. Conserv. 2015, 181, 236–244. [Google
Scholar] [CrossRef][Green Version]

5. Chandler, M.; See, L.; Copas, K.; Bonde, A.M.; López, B.C.; Danielsen, F.;
Legind, J.K.; Masinde, S.; Miller-Rushing, A.J.; Newman, G.; et al.
Contribution of citizen science towards international biodiversity monitoring.
Biol. Conserv. 2017, 213, 280–294. [Google Scholar] [CrossRef][Green
Version]

6. Couvet, D.; Jiguet, F.; Julliard, R.; Levrel, H.; Teyssedre, A. Enhancing citizen
contributions to biodiversity science and public policy. Interdiscip. Sci. Rev.
2008, 33, 95–103. [Google Scholar] [CrossRef]

7. Ballard, H.L.; Robinson, L.D.; Young, A.N.; Pauly, G.B.; Higgins, L.M.;
Johnson, R.F.; Tweddle, J.C. Contributions to conservation outcomes by
natural history museum-led citizen science: Examining evidence and next
steps. Biol. Conserv. 2017, 208, 87–97. [Google Scholar] [CrossRef][Green
Version]

51
8. McKinley, D.C.; Miller-Rushing, A.J.; Ballard, H.L.; Bonney, R.; Brown, H.;
Cook-Patton, S.C.; Evans, D.M.; French, R.A.; Parrish, J.K.; Phillips, T.B.; et
al. Citizen science can improve conservation science, natural resource
management, and environmental protection. Biol. Conserv. 2017, 208, 15–28.
[Google Scholar] [CrossRef][Green Version]

9. Irwin, E.G.; Culligan, P.J.; Fischer-Kowalski, M.; Law, K.L.; Murtugudde, R.;
Pfirman, S. Bridging barriers to advance global sustainability. Nat. Sustain.
2018, 1, 324–326. [Google Scholar] [CrossRef]

10. Nations, U. The 17 Goals|Sustainable Development. Available online:

https://sdgs.un.org/goals (accessed on 17 December 2020).

11. Fraisl, D.; Campbell, J.; See, L.; Wehn, U.; Wardlaw, J.; Gold, M.; Moorthy, I.;
Arias, R.; Piera, J.; Oliver, J.L.; et al. Mapping citizen science contributions to
the UN sustainable development goals. Sustain. Sci. 2020. [Google Scholar]
[CrossRef]

12. Fritz, S.; See, L.; Carlson, T.; Haklay, M.M.; Oliver, J.L.; Fraisl, D.; Mondardini,
R.; Brocklehurst, M.; Shanley, L.A.; Schade, S.; et al. Citizen science and the
United Nations Sustainable Development Goals. Nat. Sustain. 2019, 2,
922–930. [Google Scholar] [CrossRef]

13. ECSA. Ten Principles of Citizen Science. 2015. Available online:

https://ecsa.citizen-science.net/wp-content/uploads/2020/02/ecsa_ten_pr
inciples_of_citizen_science.pdf (accessed on 30 October 2020).

14. Robinson, L.D.; Cawthray, J.L.; West, S.E.; Bonn, A.; Ansine, J. Ten principles
of citizen science. In Citizen Science: Innovation in Open Science, Society
and Policy; UCL Press: London, UK, 2018; pp. 27–40. [Google Scholar]

15. Callaghan, C.T.; Poore, A.G.B.; Mesaglio, T.; Moles, A.T.; Nakagawa, S.;
Roberts, C.; Rowley, J.J.L.; VergÉs, A.; Wilshire, J.H.; Cornwell, W.K. Three
Frontiers for the Future of Biodiversity Research Using Citizen Science Data.
BioScience 2020, biaa133. [Google Scholar] [CrossRef]

16. Dunn, R.R.; Beasley, D.E. Democratizing evolutionary biology, lessons from
insects. Curr. Opin. Insect Sci. 2016, 18, 89–92. [Google Scholar]
[CrossRef] [PubMed][Green Version]

52
17. Ollerton, J. Pollinator Diversity: Distribution, Ecological Function, and
Conservation. Annu. Rev. Ecol. Evol. Syst. 2017, 48, 353–376. [Google
Scholar] [CrossRef][Green Version]

18. Wilson, J.S.; Forister, M.L.; Carril, O.M. Interest exceeds understanding in
public support of bee conservation. Front. Ecol. Environ. 2017, 15, 460–466.
[Google Scholar] [CrossRef][Green Version]

19. Rader, R.; Bartomeus, I.; Garibaldi, L.A.; Garratt, M.P.D.; Howlett, B.G.;
Winfree, R.; Cunningham, S.A.; Mayfield, M.M.; Arthur, A.D.; Andersson,
G.K.S.; et al. Non-bee insects are important contributors to global crop
pollination. Proc. Natl. Acad. Sci. USA 2016, 113, 146–151. [Google Scholar]
[CrossRef][Green Version]

20. Intergovernmental Science-Policy Platform on Biodiversity and Ecosystem

Services (IPBES). Assessment Report on Pollinators, Pollination and Food
Production; IPBES: Bonn, Germany, 2016. [CrossRef]

21. Patel, V.; Pauli, N.; Biggs, E.; Barbour, L.; Boruff, B. Why bees are critical for
achieving sustainable development. Ambio 2020. [Google Scholar]
[CrossRef][Green Version]

22. Barbiéri, C.; Francoy, T.M. Theoretical model for interdisciplinary analysis of
human activities: Meliponiculture as an activity that promotes sustainability.
Ambiente Soc. 2020, 23. [Google Scholar] [CrossRef]

23. Moher, D.; Liberati, A.; Tetzlaff, J.; Altman, D.G.; Group, T.P. Preferred
Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA
Statement. PLoS Med. 2009, 6, e1000097. [Google Scholar]
[CrossRef][Green Version]

24. Grames, E.M.; Stillman, A.N.; Tingley, M.W.; Elphick, C.S. An automated
approach to identifying search terms for systematic reviews using keyword
co-occurrence networks. Methods Ecol. Evol. 2019, 10, 1645–1654. [Google
Scholar] [CrossRef]

25. Haddaway, N.R.; Bernes, C.; Jonsson, B.G.; Hedlund, K. The benefits of
systematic mapping to evidence-based environmental management. Ambio
2016, 45, 613–620. [Google Scholar] [CrossRef][Green Version]

53
26. Eitzel, M.V.; Cappadonna, J.L.; Santos-Lang, C.; Duerr, R.E.; Virapongse, A.;
West, S.E.; Kyba, C.; Bowser, A.; Cooper, C.B.; Sforzi, A.; et al. Citizen
science terminology matters: Exploring key terms. Citiz. Sci. Theory Pract.
2017. [Google Scholar] [CrossRef][Green Version]

27. Grames, E.M.; Hennessy, E.A. Search Term Selection with Litsearchr for an
Example Systematic Review of the Effects of Forest Fragmentation on
Bird-Insect Interactions. Available online:
https://elizagrames.github.io/litsearchr/litsearchr_vignette_v041.html
(accessed on 23 September 2020).

28. Cooper, C.B.; Shirk, J.; Zuckerberg, B. The invisible prevalence of citizen
science in global research: Migratory birds and climate change. PLoS ONE
2014, 9, e106508. [Google Scholar] [CrossRef] [PubMed][Green Version]

29. Smith, B.M.; Chakrabarti, P.; Chatterjee, A.; Chatterjee, S.; Dey, U.K.; Dicks,
L.V.; Giri, B.; Laha, S.; Majhi, R.K.; Basu, P. Collating and validating
indigenous and local knowledge to apply multiple knowledge systems to an
environmental challenge: A case-study of pollinators in India. Biol. Conserv.
2017, 211, 20–28. [Google Scholar] [CrossRef][Green Version]

30. United Nations Statistics Division (UNSD). Tier Classification for Global SDG
Indicators. Available online:
https://unstats.un.org/sdgs/files/Tier%20Classification%20of%20SDG%2
0Indicators_17%20July%202020_web.v2.pdf (accessed on 20 October
2020).

31. Cunha, D.G.; Marques, J.F.; Resende, J.C.; Falco, P.B.; Souza, C.M.;
Loiselle, S.A. Citizen science participation in research in the environmental
sciences: Key factors related to projects’ success and longevity. An. Acad.
Bras. Ciências 2017, 89, 2229–2245. [Google Scholar] [CrossRef][Green
Version]

32. Wiggins, A.; Newman, G.; Stevenson, R.D.; Crowston, K. Mechanisms for
Data Quality and Validation in Citizen Science. In Proceedings of the 2011
IEEE Seventh International Conference on e-Science Workshops, Stockholm,
Sweden, 5–8 December 2011; pp. 14–19. [Google Scholar] [CrossRef]

54
33. Bonney, R.; Ballard, H.; Jordan, R.; McCallie, E.; Phillips, T.; Shirk, J.;
Wilderman, C.C. Public Participation in Scientific Research: Defining the Field
and Assessing Its Potential for Informal Science Education. A CAISE Inquiry
Group Report, Online Submission. 2009.

34. Kelemen-Finan, J.; Scheuch, M.; Winter, S. Contributions from citizen science
to science education: An examination of a biodiversity citizen science project
with schools in Central Europe. Int. J. Sci. Educ. 2018, 40, 2078–2098.
[Google Scholar] [CrossRef]

35. South, A. Package ‘Rworldmap’. Available online:

https://cran.r-project.org/web/packages/rworldmap/rworldmap.pdf
(accessed on 14 October 2020).

36. Blackawton, P.; Airzee, S.; Allen, A.; Baker, S.; Berrow, A.; Blair, C.; Churchill,
M.; Coles, J.; Cumming, R.J.; Fraquelli, L.; et al. Blackawton bees. Biol. Lett.
2011, 7, 168–172. [Google Scholar] [CrossRef]

37. Follett, R.; Strezov, V. An Analysis of Citizen Science Based Research: Usage
and Publication Patterns. PLoS ONE 2015, 10, e143687. [Google Scholar]
[CrossRef][Green Version]

38. van Vierssen Trip, N.; MacPhail, V.J.; Colla, S.R.; Olivastri, B. Examining the
public’s awareness of bee (Hymenoptera: Apoidae: Anthophila) conservation
in Canada. Conserv. Sci. Pract. 2020, e293. [Google Scholar] [CrossRef]

39. MacPhail, V.J.; Gibson, S.D.; Colla, S.R. Community science participants gain
environmental awareness and contribute high quality data but improvements
are needed: Insights from Bumble Bee Watch. PeerJ 2020, 8, e9141. [Google
Scholar] [CrossRef]

40. Kremen, C.; Williams, N.M.; Thorp, R.W. Crop pollination from native bees at
risk from agricultural intensification. Proc. Natl. Acad. Sci. USA 2002, 99,
16812–16816. [Google Scholar] [CrossRef] [PubMed][Green Version]

41. Billaud, O.; Vermeersch, R.L.; Porcher, E. Citizen science involving farmers as
a means to document temporal trends in farmland biodiversity and relate them
to agricultural practices. J. Appl. Ecol. 2020. [Google Scholar] [CrossRef]

55
42. Garratt, M.; Potts, S.; Banks, G.; Hawes, C.; Breeze, T.; O’Connor, R.; Carvell,
C. Capacity and willingness of farmers and citizen scientists to monitor crop
pollinators and pollination services. Glob. Ecol. Conserv. 2019, 20, e00781.
[Google Scholar] [CrossRef]

43. Appenfeller, L.R.; Lloyd, S.; Szendrei, Z. Citizen science improves our
understanding of the impact of soil management on wild pollinator abundance
in agroecosystems. PLoS ONE 2020, 15, e230007. [Google Scholar]
[CrossRef] [PubMed][Green Version]

44. Cronje, R.; Rohlinger, S.; Crall, A.; Newman, G. Does Participation in Citizen
Science Improve Scientific Literacy? A Study to Compare Assessment
Methods. Appl. Environ. Educ. Commun. 2011, 10, 135–145. [Google
Scholar] [CrossRef]

45. Queiruga-Dios, M.Á.; López-Iñesta, E.; Diez-Ojeda, M.; Sáiz-Manzanares,

M.C.; Vázquez Dorrío, J.B. Citizen Science for Scientific Literacy and the
Attainment of Sustainable Development Goals in Formal Education.
Sustainability 2020, 12, 4283. [Google Scholar] [CrossRef]

46. Everaars, J.; Strohbach, M.W.; Gruber, B.; Dormann, C.F. Microsite conditions
dominate habitat selection of the red mason bee (Osmia bicornis,
Hymenoptera: Megachilidae) in an urban environment: A case study from
Leipzig, Germany. Landsc. Urban Plan. 2011, 103, 15–23. [Google Scholar]
[CrossRef]

47. Desaegher, J.; Nadot, S.; Fontaine, C.; Colas, B. Floral morphology as the
main driver of flower-feeding insect occurrences in the Paris region. Urban
Ecosyst. 2018, 21, 585–598. [Google Scholar] [CrossRef]

48. Beckham, J.L.; Atkinson, S. An updated understanding of Texas bumble bee

(Hymenoptera: Apidae) species presence and potential distributions in Texas,
USA. PeerJ 2017, 5, e3612. [Google Scholar] [CrossRef][Green Version]

49. Suzuki-Ohno, Y.; Yokoyama, J.; Nakashizuka, T.; Kawata, M. Utilization of

photographs taken by citizens for estimating bumblebee distributions. Sci.
Rep. 2017, 7, 11215. [Google Scholar] [CrossRef]

56
50. MacPhail, V.J.; Richardson, L.L.; Colla, S.R. Incorporating citizen science,
museum specimens, and field work into the assessment of extinction risk of
the American Bumble bee (Bombus pensylvanicus De Geer 1773) in Canada.
J. Insect Conserv. 2019, 23, 597–611. [Google Scholar] [CrossRef][Green
Version]

51. Looney, C.; Strange, J.P.; Freeman, M.; Jennings, D. The expanding Pacific
Northwest range of Bombus impatiens Cresson and its establishment in
Washington State. Biol. Invasions 2019, 21, 1879–1885. [Google Scholar]
[CrossRef]

52. Prendergast, K.S. First records of the introduced African carder bee,
Pseudoanthidium (Immanthidium) repetitum (Hymenoptera: Megachilidae), in
Western Australia. Pac. Conserv. Biol. 2020. [Google Scholar] [CrossRef]

53. Birkin, L.; Goulson, D. Using citizen science to monitor pollination services.
Ecol. Entomol. 2015, 40, 3–11. [Google Scholar] [CrossRef][Green Version]

54. Brodschneider, R.; Gratzer, K.; Kalcher-Sommersguter, E.; Heigl, H.; Auer, W.;
Moosbeckhofer, R.; Crailsheim, K. A citizen science supported study on
seasonal diversity and monoflorality of pollen collected by honey bees in
Austria. Sci. Rep. 2019, 9, 1–12. [Google Scholar] [CrossRef]

55. Michener, C.D. The Bees of the World; JHU Press: Baltimore, MD, USA,
2000; Volume 1. [Google Scholar]

56. Gray, A.; Adjlane, N.; Arab, A.; Ballis, A.; Brusbardis, V.; Charrière, J.D.;
Chlebo, R.; Coffey, M.F.; Cornelissen, B.; Amaro da Costa, C.; et al. Honey
bee colony winter loss rates for 35 countries participating in the COLOSS
survey for winter 2018–2019, and the effects of a new queen on the risk of
colony winter loss. J. Apic. Res. 2020, 59, 744–751. [Google Scholar]
[CrossRef]

57. Pocock, M.J.; Chapman, D.S.; Sheppard, L.J.; Roy, H.E. Choosing and Using
Citizen Science: A Guide to When and How to Use Citizen Science to Monitor
Biodiversity and the Environment; NERC/Centre for Ecology & Hydrology:
Oxfordshire, UK, 2014. [Google Scholar]

58. Martens, P. Sustainability: Science or fiction? Sustain. Sci. Pract. Policy 2006,
2, 36–41. [Google Scholar] [CrossRef][Green Version]

57
59. Potts, S.G.; Imperatriz-Fonseca, V.; Ngo, H.T.; Aizen, M.A.; Biesmeijer, J.C.;
Breeze, T.D.; Dicks, L.V.; Garibaldi, L.A.; Hill, R.; Settele, J.; et al.
Safeguarding pollinators and their values to human well-being. Nature 2016,
540, 220–229. [Google Scholar] [CrossRef]

60. Dicks, L.V.; Viana, B.; Bommarco, R.; Brosi, B.; del Coro Arizmendi, M.;
Cunningham, S.A.; Galetto, L.; Hill, R.; Lopes, A.V.; Pires, C.; et al. Ten
policies for pollinators. Science 2016, 354, 975–976. [Google Scholar]
[CrossRef][Green Version]

61. Harvey, J.A.; Heinen, R.; Armbrecht, I.; Basset, Y.; Baxter-Gilbert, J.H.;
Bezemer, T.M.; Böhm, M.; Bommarco, R.; Borges, P.A.; Cardoso, P.; et al.
International scientists formulate a roadmap for insect conservation and
recovery. Nat. Ecol. Evol. 2020, 4, 174–176. [Google Scholar] [CrossRef]

62. Science for Environmental Policy. Pollinators: Importance for Nature and
Human Well-Being, Drivers of Decline and the Need for Monitoring; Future
Brief 23; European Commission: Brussels, Belgium, 2020. [Google Scholar]

63. Requier, F.; Andersson, G.K.; Oddi, F.J.; Garibaldi, L.A. Citizen science in
developing countries: How to improve volunteer participation. Front. Ecol.
Environ. 2020, 18, 101–108. [Google Scholar] [CrossRef]

64. Kobori, H.; Dickinson, J.L.; Washitani, I.; Sakurai, R.; Amano, T.; Komatsu, N.;
Kitamura, W.; Takagawa, S.; Koyama, K.; Ogawara, T.; et al. Citizen science:
A new approach to advance ecology, education, and conservation. Ecol. Res.
2016, 31, 1–19. [Google Scholar] [CrossRef][Green Version]

65. Loos, J.; Horcea-Milcu, A.I.; Kirkland, P.; Hartel, T.; Osváth-Ferencz, M.;
Fischer, J. Challenges for biodiversity monitoring using citizen science in
transitioning social–ecological systems. J. Nat. Conserv. 2015, 26, 45–48.
[Google Scholar] [CrossRef]

66. Mertz, O.; Ravnborg, H.M.; Lövei, G.L.; Nielsen, I.; Konijnendijk, C.C.
Ecosystem services and biodiversity in developing countries. Biodivers.
Conserv. 2007, 16, 2729–2737. [Google Scholar] [CrossRef]

67. Druschke, C.G.; Seltzer, C.E. Failures of engagement: Lessons learned from
a citizen science pilot study. Appl. Environ. Educ. Commun. 2012, 11,
178–188. [Google Scholar] [CrossRef]

58
68. Deguines, N.; Julliard, R.; de Flores, M.; Fontaine, C. The Whereabouts of
Flower Visitors: Contrasting Land-Use Preferences Revealed by a
Country-Wide Survey Based on Citizen Science. PLoS ONE 2012, 7, e45822.
[Google Scholar] [CrossRef]

69. Maher, S.; Manco, F.; Ings, T.C. Using citizen science to examine the nesting
ecology of ground-nesting bees. Ecosphere 2019, 10, e02911. [Google
Scholar] [CrossRef]

70. Viana, B.; Souza, C.; Moreira, E. Why the views of Latin American Scientists
on Citizen Science as a Tool for Pollinator Monitoring and Conservation
Matter? Neotrop. Entomol. 2020, 49, 604–613. [Google Scholar] [CrossRef]

71. Bickford, D.; Posa, M.R.C.; Qie, L.; Campos-Arceiz, A.; Kudavidanage, E.P.
Science communication for biodiversity conservation. Biol. Conserv. 2012,
151, 74–76. [Google Scholar] [CrossRef]

72. Gadermaier, G.; Dörler, D.; Heigl, F.; Mayr, S.; Rüdisser, J.; Brodschneider, R.;
Marizzi, C. Peer-reviewed publishing of results from Citizen Science projects.
J. Sci. Commun. 2018, 17, L01. [Google Scholar] [CrossRef]

73. Björk, B.C.; Welling, P.; Laakso, M.; Majlender, P.; Hedlund, T.; Guðnason, G.
Open Access to the Scientific Journal Literature: Situation 2009. PLoS ONE
2010, 5, e11273. [Google Scholar] [CrossRef] [PubMed]

74. WECD. Report of the World Commission on Environment and Development:

Our Common Future. 1987. Available online:
https://sustainabledevelopment.un.org/content/documents/5987our-com
mon-future.pdf (accessed on 30 October 2020).

75. Escobar, H. In Brazil, “Useful Idiots” Protest Cuts to Research and Education.
Science 2019, 17. Available online:
https://www.sciencemag.org/news/2019/05/brazil-useful-idiots-protest-cu
ts-research-and-education (accessed on 30 October 2020). [CrossRef]

76. Guglielmi, G. Mexican science suffers under budget cuts. Nature 2019, 572,
294–295. [Google Scholar] [CrossRef] [PubMed][Green Version]

59
5 - CAPÍTULO III “Theoretical model for interdisciplinary analysis of human
activities: Meliponiculture as an activity that promotes sustainability”

Ambiente & Sociedade 23:

https://doi.org/10.1590/1809-4422asoc20190020r2vu2020L4AO

Celso Barbiéri; Tiago Mauricio Francoy

Abstract

Meliponiculture, which is the rational keeping of stingless bees, has been practiced
for centuries by pre-colonial populations and has been gaining adepts in recent
years. In addition to being an alternative for the conservation of these pollinators, it is
an activity that promotes sustainability. From a theoretical point of view, disciplinary
approaches are insufficient to understand meliponiculture, due to the multiplicity of
Domains and aspects that compose it. Thus, this manuscript proposes a theoretical
model to analyze meliponiculture in an interdisciplinary way, dealing with the
environmental, cultural, social and economic Domains. Such activity should be
encouraged through participatory and interdisciplinary public policies, integrating the
various factors involved, such as honey farmers (meliponists), scientists,
governments, companies and non-governmental organizations. Public policy
approaches to meliponiculture as an economic activity must respect the conservation
needs of the stingless bee species, and whenever possible, the social and cultural
values attributed to the activity.

Keywords:

Stingless Bees; Interdisciplinarity; Conservation Biology

Resumen

La meliponicultura, entendida como la crianza de abejas sin aguijón, es practicada

desde hace siglos por poblaciones tradicionales y viene ganando adeptos en los
últimos años. Además de ser una alternativa para conservación de esos
polinizadores, es uma actividad promotora de sostenibilidad. Abordajes disciplinares
se han mostrado insuficientes para la comprensión de la meliponicultura, dada la

60
multiplicidad de dominios y aspectos que la componen. De esa forma, proponemos
un modelo teórico para analizar la meliponicultura de forma interdisciplinar,
incorporando los dominios ambiental, cultural, social y económico. Dicha Actividad
Debe ser incentivada por medio de políticas públicas participativas e
interdisciplinarias, integrando los diversos actores involucrados, tales como
meliponicultores, científicos, gobiernos, empresas y organizaciones no
gubernamentales. Abordajes de políticas públicas para la meliponicultura como
actividad económica deben respetar las necesidades de conservación de las
especies de abejas sin aguijón y los valores sociales y culturales atribuidos a esa
actividad.

Palabras-clave:

Abejas sinaguijón; interdisciplinariedad; Biología de la conservación

Resumo

A meliponicultura, que é a criação racional de abelhas sem ferrão, é praticada há

séculos por populações tradicionais e vem ganhando adeptos nos últimos anos.
Além de ser uma alternativa para a conservação desses polinizadores, trata-se de
uma atividade promotora de sustentabilidade. Do ponto de vista teórico, abordagens
disciplinares se demonstram insuficientes para a compreensão da meliponicultura,
devido à multiplicidade de Domínios e aspectos que a compõem. Assim, propomos
um modelo teórico para analisar a meliponicultura de forma interdisciplinar, tratando
dos Domínios ambiental, cultural, social e econômico. Tal atividade deve ser
incentivada por meio de políticas públicas participativas e interdisciplinares,
integrando os diversos atores envolvidos, como meliponicultores, cientistas,
governos, empresas e organizações não governamentais. Abordagens de políticas
públicas para a meliponicultura enquanto atividade econômica devem respeitar as
necessidades de conservação das espécies de abelhas sem ferrão e, sempre que
possível, os valores sociais e culturais atribuídos à atividade.

Palavras-chave:

Abelhas sem Ferrão; Interdisciplinaridade; Biologia da conservação

61
1. Introduction

Pollinating organisms play a fundamental role in the maintenance of global

biodiversity, officially recognized after the approval of the Convention on Biological
Diversity, of which Brazil is a signatory. The convention was part of the United
Nations Conference on the Environment (Eco-92) held in Rio de Janeiro, in 1992. In
Brazil, the text of the Convention on Biological Diversity was transformed into an
official decree, in 1994 (BRASIL, 1994).

Subsequently, in 1998, after a meeting involving pollination and pollinators

specialists from around the world, the document Declaration on Pollinators was
published, which was presented to the Convention on Biological Diversity for the
implementation of an International Pollinator Initiative (IPI). In the same year, the
Brazilian Pollinator Initiative was also created, as a way to reinforce the importance
of these organisms and provide better management of knowledge building and
spreading (IMPERATRIZ-FONSECA et al. 2007).

About 75% of the more than 240,000 species of angiosperms in the world
depend, for their reproduction, on pollinating agents, such as wind, water, animals
such as some birds, bats, insects, and others. Among them, bees are the most
important pollinators, since they are responsible for the pollination of approximately
73% of the species cultivated in the world (FREITAS, 2004). According to Gallai et
al. (2009), the total economic value of world pollination corresponded to the
equivalent of 9.5% of the value generated by agricultural production used for human
consumption in 2005. The global economic value of pollination was calculated by the
IPBES (Pollinator, Pollination and Food Production Evaluation Report) of the
Intergovernmental Science-Policy Platform on Biodiversity and Ecosystem Services,
estimated at somewhere between US $ 235 billion and US $ 577 billion (POTTS et
al. in IPBES 2016). In this sense, bees have enormous ecological and economic
importance, both for the conservation of the majority of native plant species, as for
their use in pollination programs applied to crops (MESQUITA, 2009), so that they
need special attention in the field of conservation (SILVEIRA et al., 2006).

Among the various groups of bees, we find the Meliponini tribe, whose
species are popularly known as indigenous stingless bees, due to the fact that they
have an atrophied vestigial sting. They do not, however, lose the ability to defend

62
their nests, possessing various defensive strategies for this purpose, from wrapping
themselves in hair, nibbling on the aggressor’s skin, entering body cavities such as
mouth, ears, and nostrils, or depositing plant resins on enemies. The less defensive
species, on the other hand, protect their nests building them in places of difficult
access, such as in anthills or close to nests of other more defensive bee species,
gaining protection (OLIVEIRA et al., 2013).

The rational keeping of these bees is called meliponiculture, a term coined in

1953 by researcher Paulo Nogueira Neto, one of the precursors of the stingless bees
studies in Brazil. Venturieri et al. (2007) restricted the term a little more, defining
meliponiculture as the keeping of stingless bees associated with the species that
produce and store more honey. However, regardless of the definition, this is an
ancient practice from before the colonization of the Americas by the Europeans. The
products of stingless bees and, in some cases, their creation were part of the
socio-cultural customs, including food, medicinal, ritualistic, and commercial aspects
of many Amerindian societies (PALAZUELOS-BALÍVAM, 2008).

They are also important for religious, cultural, and economic aspects.
Because they store considerable amounts of honey and pollen in their nests, and
because they produce a large quantity of cerumen, some species of stingless bees
represent an essential source or supplement of income for many families
(CORTOPASSI-LAURINO et al., 2006).

The decline of bee populations

Reports indicate that stingless bees are suffering a process of population

decline, caused mainly by the deforestation of native forests, the preferred nesting
environment for the vast majority of these species (LOPES et al., 2005). Kerr et al.
(2010) also point to deforestation, burning, and the expansion of urban areas in
regions that used to be native forests as causing the reduction of pollinator
biodiversity, as they impact their food resources and nesting sites.

Currently, the indiscriminate use of pesticides is one of the leading causes of

the decline in bee populations. There is evidence of lethal and sublethal effects, such
as malformation of the larvae, reduced life expectancy, and wing asymmetry of
Scaptotrigona aff depilis (ROSA et al., 2016). This species is indicated as a good

63
indicator of pesticide contamination (ROSA et al., 2015). It is worth mentioning that
not only pesticides have been presenting adverse effects on bee populations, but
also fertilizers, which impact both the survival and the flight behavior of Friesella
schrottkyi (RODRIGUES et al. 2017), and can also have the same effect on other
species. Thus, meliponiculture, as a colony maintenance tool, has excellent potential
for the conservation of native stingless bees, since it stimulates the maintenance of
nests and the multiplication of swarms.

2. Sustainability

Several authors have already discussed the concepts of sustainability and

sustainable development, such as the classic definition of the Brundtland report
named Our Common Future. However, although there are hegemonic approaches to
the definition of sustainability, we understand that it is more a mosaic of concepts,
which relates its multiple aspects, than a simple and unique concept.

Martens (2006) identified three fundamental characteristics of sustainability.

The first indicates that sustainability is an intergenerational phenomenon. It means
that for something to be considered sustainable, it must last longer than the time of a
generation. This duration is a necessary characteristic because, by providing comfort
and prosperity for humanity, even with certain conservation of natural resources, it is
not a sustainable situation if it does not last over time. The second common
characteristic is the level of scale. Sustainability must be practiced at several levels,
ranging from global to regional and local. What can be seen as sustainable at the
national level is not necessarily sustainable at the international level. This
geographic incompatibility derives from different interests of nations, so that it can
result in negative consequences for a given country or region. The third common
characteristic is that of multiple Domains. Sustainable development consists of at
least three domains: economic, ecological, and socio-cultural.

According to Kates (2011), the science of sustainability has the challenge of

conducting dialogues between different areas of knowledge, breaking geographical,
social and methodological barriers, in order to, finally, guide public and business
policies at regional, national and international scales.

64
 Farley (2010) presents a metaphor, in which he compares the problems
related to the loss of biodiversity and its conservation with the composed eyes of a
dragonfly (or a bee): each compound eye captures a part, or aspect, of a larger
image. This metaphor can be expanded to the discussion on sustainability as a
whole, and applied to the understanding of meliponiculture activity, which is,
according to Venturieri et al. (2003), ecologically sustainable, economically viable,
and socially fair. Besides, those three characteristics contemplate the classic tripod
of sustainability proposed by John Elkington, in 1994. However, it is important to
point out that the widely criticized (and called for recall by the author, in 2018, in
order to rethink the discussion on corporate sustainability) tripod of sustainability is
insufficient to analyze the characteristics and benefits of meliponiculture to the
present and future times.

OBJECTIVES

Given this multiplicity of Domains, we propose a theoretical model to facilitate

the understanding of activities through the prism of sustainability, with
meliponiculture being both the example and the object of analysis. Furthermore,
before starting the analysis itself, it should be noted that the proposed format and the
identified Domains are not intended to be a definitive way of analyzing
meliponiculture or any activity that uses natural resources, but rather to provide a
tool to structure the analysis and expand its understanding. Thus, we seek to
contribute both with the knowledge about the activity of meliponiculture, as well as
with the research methodologies on sustainability applied to economic activities that
use natural resources and ecosystem services.

METHODOLOGY

In a reinterpretation of the compound eyes approach, for a better

understanding of the meliponiculture activity, analysis at different levels must be
made, as it involves multiple Domains, which can be discussed separately or
together, in broad or more restricted contexts.

The following analysis was made based on the available literature on

meliponiculture, and exploratory research carried out by the authors in events

65
focused on the activity between 2014 and 2019, including participation in
meliponiculture seminars and meliponists meetings in Southeastern Brazil.
Information was collected both from discussions at events on meliponiculture and
meliponists meetings, as well as from open interviews with meliponists. Observations
were also included, made during interviews for quantitative research on
meliponiculture, conducted by the same authors of this manuscript. Spontaneous
reports of interviewed meliponists are also part of the content presented for the
demonstration of the theoretical model.

3. Meliponiculture and Sustainability

Although meliponiculture practice is a simple activity, the factors involved are

complex, with a myriad of interactions between the human hand and the
environment. For the present work, the main Domains identified are Environmental,
Social, Cultural, and Economical.

Economic Domain

Meliponiculture has been going through a recent process of popularization,

with several initiatives using this activity as a means of production. Here we identify
the main economic aspects of meliponiculture (Figure 1).

66
Figure 1 : Economical Domain of Meliponiculture

Bee products - The direct products of the bees correspond to natural

resources that are collected or produced by them, and that can be exploited
commercially. Among these resources, there are honey, pollen, propolis, cerumen (a
mixture of wax and propolis made by stingless bees), in addition to the bee colonies
themselves. In the last decade, the search for new honey flavors, mainly on the part
of the haute cuisine sector, has stimulated the activity of meliponiculture, generating
an interest increase in relation to the kinds of honey produced by native bees, due to
the disclosure of these products by internationally renowned chefs (VILLAS-BÔAS,
2017). The use of honey is extensive and can be consumed fresh or as part of sweet
and savory recipes. Due to the diversity of bee species and floral origins, these
honeys are often compared to wines, given their unique sensory characteristics. The
pollen of stingless bees has also been gaining space in haute cuisine, being used
mainly as a seasoning, due to its intense flavors. As for cerumen, some meliponists
have been testing and developing new products, such as aromatic candles (aroma
from the propolis contained in the product), handmade cosmetics as lip balms
(VILLAS-BÔAS, 2018), and even alternatives to the use of paraffin on surfboards.
Propolis, on the other hand, shows enormous potential in the pharmacological

67
sector, having already shown antibiotic effects (MAIA-FILHO, 2008), besides the
inhibition of certain types of cancer in animal models (ARAÚJO, 2010). The interest
in bee colonies is mainly due to the expansion and popularization of the activity in
Brazil, which consequently increases the demand for colonies. Thus, the production
of colonies from the induced rational multiplication may also be explored as a form of
economic gain.

Products for meliponiculture - In recent years, virtual stores have appeared,

dedicated only to the commercialization of products for the meliponiculture activity.
Those products are as follows: attractive scents for swarms, rational hives, tools,
trap nests, nests, and other miscellaneous items, which are widely marketed by
meliponists and for meliponists. The sales of these and related items is also an
alternative option of economic gain within the field of meliponiculture.

Agricultural pollination - The pollination carried out by stingless bees, directed to

specific cultures, as is already done by Australian stingless beekeepers
(HALCROFT, 2013), is an activity with enormous potential that is not yet well
explored in Brazil. Only in recent years, it has been treated with more attention by
initiatives, to stimulate targeted pollination, such as the ApiAgri initiative, which
developed an application aimed at connecting farmers interested in renting bee
colonies to pollinate their crops and beekeepers (honey bee and stingless
beekeepers) who can supply these colonies.

Employment and income - According to Venturieri et al. (2003), the rational keeping
of stingless bees is an essential activity for income and employment generation,
helping to increase the life quality of the honey producer. Several initiatives have
contributed to the generation of employment and income through meliponiculture,
such as the project developed by the Instituto Socioambiental (ISA) in the Xingu
Indigenous Land (TIX), with indigenous meliponists, who sell the excess honey
produced to restaurants in São Paulo ( VILLAS-BÔAS, 2017). The Tupyguá
cooperative in Espírito Santo is also a successful case of community-based
meliponiculture that adds income to producers. In recent years, some companies
have started to sell stingless bee honey. These companies have the peculiarity of
valuing the regionalism of the produced honey. The main examples of this regard are
the companies Mbee, which sells different kinds of honey from small producers all

68
over Brazil, stimulating local production chains, and Beeliving, which sells honey
from the Atlantic Forest. In January, 2019, the Peabiru Institute started to market the
first honey from stingless native bees with the stamps of the Ministries of Agriculture,
of Livestock and Supply, and of the Environment.

Cultural Domain

The same pyramid model can be used to represent the cultural Domain (Figure 2),
which, for the case of meliponiculture, represents a link between the traditional and
the innovative; between basic education and scientific development.

Figure 2: Cultural Domain of Meliponiculture

Traditional knowledge - Traditional knowledge is the foundation pillar of

meliponiculture, since it started with the indigenous populations of the Americas and
is still maintained in some regions of Brazil and the world (CARVALHO et al., 2014).
These populations developed a very close relationship with meliponiculture, with
bees as part of their religious rites (PALAZUELOS-BALÍVAM, 2008). Some
populations, such as the Kayapós, developed their methods for classifying bee

69
species, anatomy and structure of the nests, in a very similar way to that of
contemporary science, as shown by the study by Posey and Camargo (1987). Some
of these traditional communities have an income-generating activity in
meliponiculture, mainly due to socio-environmental projects that strengthen this
community-based production chain in several regions of Brazil (VILLAS-BÔAS,
2018).

Innovation - On the other hand, meliponiculture is an activity full of innovations

and space for new ideas. Meliponists develop new techniques and tools to deal with
species diversity, behavior, and overcome particular difficulties in their management.
Although most of this innovation occurs empirically, novelties are frequently
presented on social networks and events focused on meliponiculture, being often left
aside by the academic environment.

Science - There is much space for scientific development related to

meliponiculture, stimulating this interface between the innovations coming from the
meliponists and the members of the scientific community. If the two sides worked
with a more joint approach, it would allow the absorbing, testing, and validating of the
knowledge and innovation produced by meliponists, as well as providing new
knowledge for more efficient management, helping to continue the development of
the activity sustainably. Much can still be done concerning the development of basic
research on the field of biology and management of stingless bees and applied
research for the sustainable use of bees and their products.

Environmental education - Some initiatives have been using meliponiculture

as an educational tool for teaching the themes of the importance of bees and
pollination. Some of these initiatives are the NGO Bee or not to be, the
Meliponicultura.org initiative, the project SOS Abelhas sem Ferrão in São Paulo and
the project Enxameia, which operates in Mato Grosso do Sul. According to Barbiéri
(2018), one of the main objectives of the stingless beekeeping in the state of São
Paulo is environmental education. Since 2017, meliponiculture has become a
scouting specialty, based on joint action between the SOS Abelhas sem Ferrão
project and the Brownsea First Scout Group. The entry of meliponiculture in the list
of scouting specialties represents the initial contact of thousands of children and
adolescents with the theme of bee conservation and the importance of pollination, in

70
addition to encouraging these young people to know the subject in greater depth to
obtain the specialty badge. Educational materials aimed at environmental education
with the theme of bees have also been successfully used. Some examples are the
book “The invisible link” that, through a children’s story, brings children closer to
stingless bees, and the Bee or not to Be NGO’s Environmental Education Activities
Notebook (PERUCHI and GONÇALVES 2015).

Social Domain

In the pyramid that makes up the Social Domain of meliponiculture (Figure 3),
some intangible benefits generated by the activity are highlighted.

Figure 3 : Social Domain of Meliponiculture

Leisure - Due to the low risk of accidents when compared to traditional

beekeeping, the possibility of harvesting honey and performing a series of practices
and management are described as fun and relaxing by many meliponists. Thus, the
practice of meliponiculture as a recreational activity is an essential factor in the
popularization of the activity, leading beekeepers to come into contact with a series
of fundamental values of sustainability. According to Villas-Bôas (2018), recreational

71
meliponiculture is an important step for conservationist meliponiculture, if well
oriented, in addition to promoting the conservation of several species.

Social interactions - The generation of interactions and social relationships

between beekeepers is one of the benefits generated by the activity. Meliponists
have been forming very active communities, with local and regional meetings
gathering hundreds of people, in addition to the extensive use of social networks for
discussions involving the subject. There are now social media groups with thousands
of active members. In this way, meliponiculture can, also, create bonds of friendship
between beekeepers. According to Barbiéri (2018), online social networks are the
primary vehicle for learning about the topic in the state of São Paulo. The use of
social networks and the participation of beekeepers in seminars and meetings are
significant for knowledge exchange and the popularization of the activity. It is also
important to mention that since its foundation, the NGO Bee or not to be has
promoted several meliponiculture events in Southeast Brazil, mainly in São Paulo,
providing both social interactions between meliponists and the transmission of
management techniques.

Social inclusion - Due to the low initial cost, and because it is an elementary
activity at the starting point (but that requires much knowledge at a more advanced
level), meliponiculture can be used as a tool for social inclusion and can be
promoted among communities in need, working together with familiar agricultural
activities. For meliponiculture to expand in terms of social inclusion, public policies
that unite social inclusion, regional development, and conservation must be
implemented. A notable example of social inclusion through meliponiculture is the
Heborá project, which trains low-income countryside women, many of them
considered as “settled peasants”, aiming at social inclusion and economic
independence. The project assists countryside women from the training and
assembly of the meliponary to the honey and bee’s products marketing. Additionally,
actions to popularize the activity should be encouraged, such as the provision of
courses and workshops for the population in general, but mainly for needy
communities.

Quality of life - A benefit frequently reported by honey farmers is the

improvement in quality of life, generally attributed to the adoption of sustainable

72
practices such as independent food production, greater concern with the origin and
destination of the consumed products, and the motivation generated by making
some management on bees. A lasting example of a project combining
meliponiculture and an increase in the quality of life is the Bees for the Best Age, an
extension course held annually since 2003 at USP in Ribeirão Preto. The project
aims to teach retirees a new activity based on practical and theoretical activities,
encouraging the elderly to remain intellectually active after retirement (FREITAS,
2003). There are frequent reports of meliponists who practice meliponiculture not
only as a form of leisure but also as a therapeutic resource against depression and
anxiety, a topic that deserves attention and dedicated studies in the field of
psychology.

Environmental Domain

Finally, we will address the benefits generated by meliponiculture in the

Environmental Domain (Figure 4), essential for any discussion of meliponiculture and
sustainability. The aspects related to this Domain are: Environmental Awareness,
Biological Conservation, Environmental Recovery, and Maintenance of Ecosystem
Services.

Figure 4 Environmental Domain of Meliponiculture

73
 Maintenance of ecosystem services - One of the benefits generated by
meliponiculture is the maintenance of ecosystem services, since, in the keeping of
stingless bees, meliponists maintain a large number of pollinating agents responsible
for pollination service. From this perspective, stimuli to the development of rational
meliponiculture are essential for the preservation of both stingless bees and plant
populations dependent on their pollination. According to Kerr et al. (1996), stingless
bees may be responsible for pollinating up to 30% of the plant species in the
Caatinga and Pantanal biomes, reaching up to 90% of the pollination of some
remaining fragments of the Atlantic Forest.

Environmental recovery - A peculiar aspect of meliponiculture is the promotion

of environmental recovery. A widespread practice among stingless beekeepers is to
cultivate plants that provide resources for stingless bees, such as pollen, nectar, and
resins. Due to the need to provide resources for bees, the meliponists, in addition to
promoting plantings of native plants, exchange seedlings and plant seeds that supply
these resources, improving environmental quality, even in urban environments.
Although these environments are, in general, quite degraded, they represent crucial
refuges for pollinator populations (HALL et al., 2017). When dealing with commercial
meliponiculture, we can consider environmental recovery as a positive externality of
the meliponiculture production chain.

Environmental awareness - Understanding the importance of bees and

pollination to the environment, such as the maintenance and restoration of flora, is
almost a mandatory consequence of meliponiculture. Barbiéri (2018) and Jaffé et al.
(2015) pointed out that environmental problems such as monocultures,
deforestation, use of pesticides, and the spraying of insecticides in urban areas for
the control of disease vector mosquitoes are widely perceived and considered
dangerous by meliponists. Understanding ecosystem services and the environmental
problems involved in the human-environment relationship is an essential part of
environmental awareness, which is essential for the promotion of sustainable
practices.

Biological conservation - The promotion of biological conservation

characterizes the rational keeping of stingless bees, both for the bees themselves
and for the plants dependent on their pollination. Given the advance of deforestation

74
in Brazil, and the dependence of stingless bees on nesting sites, rational
meliponaries are relevant maintainers of this biodiversity (COSTA et al., 2014). Both
recreational and professional meliponiculture present the potential to be mechanisms
of biological conservation, although, for this, adequate training and technical
references are necessary. Knowledge of management practices, seasonality, and the
keeping of local species are essential for this (VILLAS-BÔAS, 2018). As a safe
conduct, commercial meliponaries have a large number of colonies, so it is
necessary to plant melitophilous pasture that supports those populations. It is, also,
necessary to take precautions to prevent the proliferation of diseases, as well as to
avoid the keeping of non-endemic bee species. Another relevant aspect is that
commercial meliponaries are focused on a small number of species, while hobbyist
meliponists tend to have fewer colonies, but of several species, although many
stingless bee species have sophisticated or non-established management practices,
being rarely kept by meliponists.

Interdisciplinary approaches

The examples briefly discussed for each Domain are only small parts that
make up a much larger universe. If, within each Domain, we can analyze and
discuss several issues grouped or separately, the same goes for the Domains
themselves, which make up the universe of meliponiculture. This myriad of aspects
and factors must be understood in a similar way to a pyramid: when looking at one
face, it is not possible to be aware of the others, or even the base. To understand the
pyramid as a whole, it is necessary to change the point of view, in order to
understand each part with the necessary attention, yet without losing sight of the
whole. It is worth mentioning that each face, in turn, has smaller parts that,
consecutively, are composed of their own even smaller parts. However, even though
at certain times we may focus on the small parts of this fractal that forms the activity
of meliponiculture, we cannot fail to pay attention to the multiple domains that
compose it as a whole (Figure 5).

75
Figure 5 : Multiple Domains of Meliponiculture

In that way, meliponiculture and its development within the context of

conservation and sustainability must be treated by interdisciplinary factions, who can
move between micro and macro scales, from their specific areas of knowledge to
broader scales, in a way that neither contexts nor details are addressed with a lack
of quality.

Other examples of the application of the Sustainability pyramid model

To exemplify the use of the analysis model proposed here, three activities with
different levels of sustainability are listed below. Each of the activities can be
analyzed in greater depth in other works, so these specific ones are used only for a
didactic purpose, to illustrate other potential uses of the model.

76
1 - Agroforestry Systems are food production systems that value biodiversity and
seek to draw on the natural dynamics of interactions between organisms, sometimes
integrating forest management, species of food plants, and small-scale livestock.
Agroforestry Systems promote sustainability in all domains (CAPORAL &
COSTABEBER, 2002). In the Environmental Domain, it promotes Biodiversity
Conservation and Recovery, maintaining essential Ecosystem Services (soil quality,
water bodies, soil, pollinator resources) (RIBASKI, 2008; ALMEIDA, 2010;
AMADOR, 2003). In the Economic Domain, they generate Employment and Income,
moving the regional economy, supplying various forest, agricultural, and animal
products, in addition to the added value of the products due to the SAF’s
sustainability values (KITAMURA & RODRIGUES, 2001; ALMEIDA, 2010). In the
Cultural Domain, SAFs are very similar to Meliponiculture. They use and preserve
traditional knowledge, are rich in innovation, especially concerning management
techniques and species consortia, they are an excellent tool for environmental
education, as a way of reconnecting between man and nature through food. They
are also excellent research objects for Science, whether for the improvement and
development of SAFs or a better understanding of the interactions between
cultivated organisms (ALMEIDA, 2010). In the Social Domain, they promote a better
quality of life, the possibility of social inclusion, incorporating men in the countryside,
and valuing social interactions (CASTRO et al., 2009), and it can also be a form of
leisure, as community gardens do in urban environments.

2 - Apiculture, the keeping of bees of the Apis genus, originating in Europe, Asia,
and Africa. It is a well-established activity in almost the entire world, including Brazil,
which contains sustainable characteristics, although to a lesser extent than
meliponiculture. In the Environmental Domain, it promotes ecosystemic pollination
services, although with limited possibilities when compared to the use of stingless
bees (VILLAS-BÔAS, 2018). As they use exotic species for the production of honey,
the contribution to biological conservation is smaller. Among beekeepers, the
practice of migrating colonies according to the flowering is very recurrent, while the
stingless beekeepers usually plant to provide the resources for the bees. In the
Economic Domain, beekeeping is a very well-established activity, generating
employment, income, bee products, applied agricultural pollination, and a market for
tools and personal protection equipment. Nowadays, Apiculture generates more

77
income than Meliponiculture, but with less potential for creating new products with
high added value. In the Cultural Domain, beekeeping carries traditions consolidated
for centuries, but it is still an important object of scientific research, but with smaller
potential for innovation than Meliponiculture. It can also be used as an
Environmental Education tool, although in this aspect, Meliponiculture has several
advantages. In the Social Domain, beekeeping is a significant activity, valuing social
interactions between breeders, increasing the quality of life of people in the
countryside, serving as a form of social inclusion, and also as a form of leisure
(GONÇALVES, 2006).

3 - Monoculture is the extensive agriculture with very low biodiversity. Although it is

one of the main economic activities in Brazil, it is an unsustainable activity (TILMAN,
1999), due to the following reasons: In the Environmental Domain, due to the
suppression of natural areas, low biodiversity, impoverishment of the soil, reduction
of essential ecosystem services (TILMAN, 1999). On the Social Domain, due to the
risks generated to workers and consumers of their products as a result of the use of
pesticides, reduced contact with nature, less opportunity for social inclusion, and low
quality of life for rural workers (REIS & NETO, 2014). In the Cultural Domain,
although there is significant scientific and technological development, innovation is
mostly generated by the environmental unsustainability scenario, to overcome
problems generated by the system itself and which are much smaller in other
productive systems (ARAÚJO, 2014; ZIMMERMANN, 2009). In monoculture,
traditional knowledge that was widespread even before the Green Revolution was
lost (REIS & NETO, 2014), and the most significant possibility of environmental
education is the use of counterexamples, compared to other production methods. In
the Economic Domain, monocultures are considered as an important generator of
wealth, however, the model encourages the concentration of income and the
precariousness of the workforce (ARAÚJO, 2014). Furthermore, although it is the
source of many commodities and raw materials, the diversity of products generated
by monocultures is increasingly smaller (ZIMMERMANN, 2009) than other methods.

78
4. Conclusion

The Sustainability pyramid model can be applied to the analysis of any human
activity of economic interest that uses natural resources or ecosystem services.
Using this model for the analysis of meliponiculture, it is possible to observe that
public policies to encourage the activity should be conducted, in order to facilitate
their recreational and professional practice. Besides, these policies should, also,
provide technical references for the activity, in favor of developing and promoting
employment and income combined with the conservation and sustainable use of
biodiversity. The implementation of public policies for meliponiculture requires a
plural debate with a view to sustainability at local, regional, and global scales. Such
factors create an opportunity for the development of research involving stingless
beekeepers as an active part of knowledge production in participatory programs that
bring together scientists, companies, honey producers, stakeholders, and other
sectors involved in the meliponiculture activity.

Due to the multiplicity of aspects and benefits generated by meliponiculture in

the environmental, social, cultural, and economic Domains, non-interdisciplinary
approaches are not sufficient to understand meliponiculture, which is an activity that
promotes sustainability. Thus, approaches aimed at the elaboration of public policies
for meliponiculture as an economic activity must respect the needs for the
conservation of stingless bee species, and whenever possible, the social and cultural
values attributed to the activity.

References

● ALMEIDA, RG de, et al. Sistemas agrossilvipastoris: benefícios técnicos,

econômicos, ambientais e sociais. ENCONTRO SOBRE ZOOTECNIA DE
MATO GROSSO DO SUL, 2010, 7: 1-10.

● Amador, D. B. (2003). Restauração de ecossistemas com sistemas

agroflorestais. Restauração de ecossistemas naturais. Fundação de Estudos
e Pesquisas Agrícolas e Florestais-FEPAF. São Paulo. Botucatu.

79
● ARAÚJO, M. J; AM et al. Effect of propolis of Scaptotrigona aff. postica on the
development of the tumor of Ehrlich in mice. Revista Brasileira de
Farmacognosia, v. 20, n. 4, p. 580-587, 2010.

● BRASIL. Decreto Legislativo no 2, de 3 de fevereiro de 1994. Aprova o texto

da Convenção sobre Diversidade Biológica. Diário do Congresso Nacional
(Seção II) Brasília, DF, 8 de fevereiro de 1994, pp. 500-510.

● CASTRO, Albejamere Pereira de, et al. Os sistemas agroflorestais como

alternativa de sustentabilidade em ecossistemas de várzea no Amazonas.
Acta Amazonica, 2009, 39.2: 279-288.

● CAPORAL, Francisco Roberto; COSTABEBER, José Antônio. Análise

multidimensional da sustentabilidade. Agroecologia e desenvolvimento rural
sustentável, 2002, 3.3: 70-85.

● COSTA, L. et al. Rescue of Stingless bee (Hymenoptera: Apidae: Meliponini)

nests: an important form of mitigating impacts caused by deforestation.
Sociobiology, v. 61, n. 4, p. 554-559, 2014.

● DE ARAUJO, Rodrigo da Cruz. Análise sobre a monocultura de soja e o

Desenvolvimento Sustentável na Amazônia com base na teoria do
Desenvolvimento Endógeno. Economia e Desenvolvimento, 2014, 26.1.

● FREITAS, B. M.; PEREIRA, J. O. P. Conservation and management of

pollinators for sustainable agriculture: The international response. InA
contribution to the International Workshop on solitary bees and their role in
pollination held in Beberibe, Ceará, Brazil. Solitary bees: Conservation,
rearing and management for pollination. Fortaleza: Imprensa Universitária,
2004.p.42-58

80
● FREITAS, G.S. et al. Abelhas para a melhor idade: curso de meliponíneos,
alfabetização técnica para a conservação. Bioscience Journal, v. 23, 2007.

● GONÇALVES, L. S. Meio século de apicultura com abelhas africanizadas no

Brasil. Mensagem Doce, v. 87, n. 1, p. 21-26, 2006.

● HALL, D. M. et al. The city as a refuge for insect pollinators. Conservation

Biology, v. 31, n. 1, p. 24-29, 2017.

● HALCROFT, M. T. et al. The Australian stingless bee industry: a follow-up

survey, one decade on. Journal of Apicultural Research, v. 52, n. 2, p. 1-7,
2013.

● IMPERATRIZ-FONSECA, V. L.; SARAIVA, A. M.; GONÇALVES, L.S. A

iniciativa brasileira de polinizadores e os avanços atuais para a compreensão
do papel dos polinizadores como produtores de serviços ambientais.
Bioscience Journal, v. 23, 2007.

● KERR, W. K.; CARVALHO, G. A.; NASCIMENTO, V. A. Abelha Uruçu:

Biologia, Manejo e Conservação. Paracatú: Acangaú, 1996. 144 p.

● KERR, Warwick Estevam et al. Aspectos pouco mencionados da

biodiversidade amazônica. Parcerias Estratégicas, v. 6, n. 12, p. 20-41, 2010.

● KITAMURA, Paulo Choji; RODRIGUES, Geraldo Stachetti. Valoração de

serviços ambientais em sistemas agroflorestais: métodos, problemas e
perspectivas. In: III Congresso Brasileiro de Sistemas Agroflorestais. 2001. p.
55.

● LOPES, Marcio; FERREIRA, João Batista; DOS SANTOS, Gilberto. Abelhas

sem-ferrão. APA Agriculturas, v. 2 - n 4 , 2005.

81
● MAIA-FILHO, E. M. et al. Efeito antimicrobiano in vitro de diferentes
medicações endodônticas e própolis sobre Enterococcus faecalis. RGO, v.
56, n. 1, p. 21-25, 2008.

● MBEE MEL DE TERROIR. Disponível em: <http://mbee.com.br/index.html>.

Acesso em: 03 de fev. 2019.

● MESQUITA, T. M. S. Diversidade de abelhas solitárias (Hymenoptera,

Apoidea) que nidificam em ninhos-armadilha em áreas de Cerrado, MG.
2009. 50 f. Dissertação (Mestrado em Entomologia) - Universidade Federal
de Uberlândia, Uberlândia, 2009.

● OLIVEIRA, F. F. et al .Guia Ilustrado das Abelhas Sem-Ferrão das Reservas

Amanã e Mamirauá, Amazonas, Brasil (Hymenoptera, Apidae, Meliponini).
Tefé: Instituto de Desenvolvimento Sustentável Mamirauá IDSM/OS/MCTI,
2013.

● PALAZUELOS BALLIVIAN, J. M. P.; UTERMOEHL B. ; SOARES, V. M. .

Abelhas nativas sem ferrão. 2. ed. São Leopoldo: OIKOS, 2008. v. 1. 128p.

● PERUCHI, Rosane M. G.; GONÇALVES, L. S.; .SEM ABELHA SEM

ALIMENTO: Caderno de Atividades para Educação Ambiental. 1a.. ed. Rio de
Janeiro: FUNBIO, 2015. v. 1. 56p.

● POSEY, D. A.; CAMARGO, J.M.F. Additional notes on the classification and

knowledge of stingless bees (Meliponinae, Apidae, Hymenoptera) by
Kayapóindians of Gorotire, Pará, Brazil. Annals of Carnegie Museum, v. 54, n.
8, p. 247-274, 1985.

● POTTS, Simon G. et al. The assessment report of the Intergovernmental

Science-Policy Platform on Biodiversity and Ecosystem Services on
pollinators, pollination and food production. 2016.

82
● Reis, L. C. S., & Neto, J. D. C. G. (2014). Produção de alimentos: agricultura
familiar x cultura de exportação no Brasil, sob a perspectiva da
sustentabilidade. Revista Brasileira de Políticas Públicas, 4(1).

● RIBASKI, Jorge. Sistemas agroflorestais: benefícios socioeconômicos e

ambientais. In: Embrapa Florestas-Artigo em anais de congresso (ALICE). In:
SIMPÓSIO SOBRE REFLORESTAMENTO NA REGIÃO SUDOESTE DA
BAHIA, 2., 2005, Vitória da Conquista. Memórias. Colombo: Embrapa
Florestas, 2008. p. 89-101.

● RODRIGUES, C. G. et al. Leaf fertilizers affect survival and behavior of the

Neotropical stingless bee Friesellaschrottkyi (Meliponini: Apidae:
Hymenoptera). Journal of economic entomology, v. 109, n. 3, p. 1001-1008,
2016.

● ROSA, A. S., et al. Consumption of the neonicotinoid thiamethoxam during

the larval stage affects the survival and development of the stingless bee,
Scaptotrigonaaff. depilis. Apidologie, v. 47, n. 6, p. 729-738, 2016.

● ROSA, A.S, et al. The stingless bee species, Scaptotrigonaaff. depilis, as a

potential indicator of environmental pesticide contamination. Environmental
toxicology and chemistry, v. 34, n. 8, p. 1851-1853, 2015.

● SILVEIRA, F. A. et al. Taxonomic constraints for the conservation and

sustainable use of wild pollinators-the Brazilian wild bees. Pollinating bees
-the conservation link between agriculture and nature. Brasilia, Ministry of
Environment, p. 41-50, 2002.

● SOS ABELHAS SEM FERRÃO. Institucional. Disponível em:

<http://sosabelhassemferrao.com.br/site/>. Acesso em: 04 de fev 2019.

83
● TILMAN, David. Global environmental impacts of agricultural expansion: the
need for sustainable and efficient practices. Proceedings of the National
Academy of Sciences, 1999, 96.11: 5995-6000.

● VENTURIERI, G.C. et al. Caracterização, colheita, conservação e

embalagem de méis de abelhas indígenas sem ferrão. Belém: Embrapa
Amazônia Oriental, 2007

● VENTURIERI, G. C. et al. Caracterização e avaliação de abelhas indígenas e

de plantas melíferas utilizadas para a produção de mel, entre os pequenos
agricultores da Amazônia Oriental. Relatório de Pesquisa. Belém: EMBRAPA,
2003.

● VILLAS-BÔAS, J. K. As abelhas nativas e a experiência da meliponicultura.In

VILLAS-BÔAS, A. et al.(Org) XINGU : Histórias dos produtos da floresta. São
Paulo. Instituto Socioambiental, 2017.p 93-120.

● VILLAS-BÔAS, J. K. Manual Tecnológico de Aproveitamento Integral dos

Produtos das Abelhas Nativas Sem Ferrão. Brasília - DF. Instituto Sociedade,
População e Natureza (ISPN). 2a edição. Brasil, 2018.

● ZIMMERMANN, Cirlene Luiza. Monocultura e transgenia: impactos

ambientais e insegurança alimentar. Veredas do Direito: Direito Ambiental e
Desenvolvimento Sustentável, 2011, 6.12.

84
6 - CAPÍTULO IV Data Reliability in a Citizen Science Protocol for Monitoring
Stingless Bees Flight Activity

Insects 2021, 12(9), 766; https://doi.org/10.3390/insects12090766

by: Jailson N. Leocadio 1,*, Natalia P. Ghilardi-Lopes 2,Sheina Koffler 3, Celso Barbiéri 4,
Tiago M. Francoy 4, Bruno Albertini 1 and Antonio M. Saraiva 1,3

1 Escola Politécnica, University of São Paulo, Av. Prof. Luciano Gualberto 158, Tv. 3, São Paulo
05508-010, SP, Brazil; 2 Centro de Ciências Naturais e Humanas, Federal University of ABC, R.
Arcturus 3, São Bernardo do Campo 09606-070, SP, Brazil; 3 Instituto de Estudos Avançados,
University of São Paulo, R. Praça do Relógio 109, São Paulo 05508-970, SP, Brazil; 4 Escola de
Artes, Ciências e Humanidades, University of São Paulo, R. Arlindo Bettio 1000, São Paulo
03828-000, SP, Brazil*

Author to whom correspondence should be addressed.

Simple Summary

This work aims to validate a citizen science protocol for monitoring the flight activity
of stingless bees. The count of flight activity (entrance, exit, and entrance carrying
pollen) filmed in 30 s videos was compared among three different groups: “original”
citizen scientists (group that filmed and performed the count in their own videos),
“replicator” citizen scientists (group of citizen scientists who performed flight activity
counts on videos shot by other citizen scientists), and experts (researchers who work
with bees and who performed the counts on videos shot by citizen scientists). The
analysis was divided into two levels: perception (detection of activity in videos) and
counting. The results of this analysis revealed that citizen scientists and experts
have similar perception and count of bee entrance and exit activity, as no statistical
differences were found in these two items. However, replicator citizen scientists
noticed more bees carrying pollen than original citizen scientists and experts.
Despite this, considering only the videos in which the groups agreed on the presence
of pollen, the count was similar for both. These results enabled the validation of the
protocol and indicated high quality of data produced by individuals who participate in
scientific practices following a citizen science approach.

85
Abstract

Although the quality of citizen science (CS) data is often a concern, evidence for
high-quality CS data increases in the scientific literature. This study aimed to assess
the data reliability of a structured CS protocol for monitoring stingless bees’ flight
activity. We tested (1) data accuracy for replication among volunteers and for expert
validation and (2) precision, comparing dispersion between citizen scientists and
expert data. Two distinct activity dimensions were considered: (a) perception of flight
activity and (b) flight activity counts (entrances, exits, and pollen load). No significant
differences were found among groups regarding entrances and exits. However,
replicator citizen scientists presented a higher chance of perceiving pollen than
original data collectors and experts, likely a false positive. For those videos in which
there was an agreement about pollen presence, the effective pollen counts were
similar (with higher dispersion for citizen scientists), indicating the reliability of
CS-collected data. The quality of the videos, a potential source of variance, did not
influence the results. Increasing practical training could be an alternative to improve
pollen data quality. Our study shows that CS provides reliable data for monitoring
bee activity and highlights the relevance of a multi-dimensional approach for
assessing CS data quality.
Keywords: biodiversity monitoring; data quality; meliponini; protocol validation; volunteer
participation

Graphical abstract, art by Paula Drago

86
1. Introduction

Citizen science (CS) is a scientific approach that allows members of the general
public to contribute to the scientific process, usually as data collectors but desirably
in other scientific inquiry steps [1,2]. As data contributors, the participants can
provide large amounts of data that, otherwise, would require great time availability,
as well as substantial financial resources [3,4]. This scattered and diverse
information can reduce data scarcity problems, which are common in some study
fields (e.g., species populations distribution, water quality monitoring). Albeit still
underutilized, CS can be a valuable resource for global change research and UN
Sustainable Development Goals achievement [3,5].

Data quality (DQ), which can be considered a multi-dimensional issue [6], is a

concern for several researchers involved with CS activities [7,8]). Studies report on
different dimensions of data quality, such as standardized sampling (e.g., [9]), spatial
and temporal representativeness and bias (e.g., [10]), data accuracy and precision
[8,11,12], sample size (e.g., [13,14,15,16]), volunteers’ proper training (e.g.,
[17,18,19]), and the experience and ability levels of participants [20,21].

In this sense, the work developed by Wiggins and Crowston [2] analyzed the
mechanisms for data quality assurance in 128 CS projects and concluded that the
topic is a concern in most of them. The most common procedures to ensure data
quality observed were: expert review, photo submissions, paper data sheets
submitted along with online entry, replication or rating by multiple participants, quality
assurance/quality control (QA/QC), training programs, and automatic filtering of
unusual reports. Pilot-testing of citizen science protocols is also a strategy commonly
used to improve data quality and reliability. The feedback from testing participants is
essential to redesign the protocols and build appropriate materials for the project
[22].

Strategies for DQ control may be applied during data collection, data

classification, or data analyses through statistical and modeling tools [8]. Data quality
control during classification, for instance, may include replication among volunteers
(distinct volunteers performing the same task and reaching some
consensus—cross-comparisons) and expert validation (comparing citizen scientists’

87
data with professional scientists’ data) [23]. However, few studies have
systematically tested these control mechanisms (e.g., [16,24,25,26]. Aceves-Bueno
et al. [27], for instance, analyzed 63 citizen science papers that reported 1363
observations of expert validation and found that 73% of the abstracts described the
contributions of citizen science as positive (accurate, reliable, comparable,
statistically similar, or valuable) and only 13% assessed citizen scientists’ (cs)
performance negatively (no significant correlations, overestimated, or
contradictions). In addition, validated CS data was reported to be more cost-effective
than traditional methods [15].

In a review of citizen science initiatives with bees, Koffler et al. [28] reported the
use of various strategies related to data quality assessment and control, mainly
digital vouchers (photographs submitted by citizen scientists) (43.2%), expert review
of data (40.9%), use of structured protocols (40.9%), and training of participants
(29.6%). The same initiative used up to five different strategies, indicating that data
reliability was a major concern for the projects’ teams. For instance, data quality was
stated as the primary objective by 13.6% of the 88 studies analyzed, with protocols
mainly related to sampling effort and species identification. While bumblebees and
honey bees were the most investigated groups, only three works studied stingless
bees, despite the increasing interest in this group due to their importance as
pollinators [29] and the global expansion of meliponiculture activities [30,31].
Stingless bees comprise a diverse group with more than 500 recognized species in
tropical and subtropical regions [32] and stingless beekeeping may be an important
tool for sustainable rural development and conservation [33]. However, the lack of
basic knowledge of stingless bee ecology and management still hampers the
practice [31,34]. In this context, beekeepers may act as important partners in CS
projects with stingless bees, following successful ongoing initiatives with honey bees
[35]. Monitoring stingless bees’ flight activity, for instance, can help us understand
several factors that affect colony performance, such as responses of foraging bees
to intra-colony stimuli and meteorological conditions. Flight activity data also serve
as an economic evaluation of the colony since the number of foraging trips is directly
linked to colony production and pollination services [36]. Therefore, good quality data
production is essential to subsidize management strategies.

88
 The present study aimed to assess the data reliability of a structured citizen
science protocol for monitoring stingless bee flight activity. Our initial hypothesis is
that there are no statistical differences between the data produced by citizen
scientists in comparison to data produced by experts, although the dispersion of the
data produced by citizen scientists may be greater. Participants of an outreach
course related to citizen science and meliponiculture produced the data. Original
data gathered by citizen scientists was first replicated by a group of citizen scientists,
who also participated in the project, and then validated by a group of experts. We
tested data accuracy for replication (comparing original and replicated data) and for
expert validation (comparing citizen scientists and expert data). Moreover, precision
was analyzed for data validation, comparing dispersion between replicator citizen
scientists and expert data. Since flight monitoring is a task with considerable
difficulties for untrained personnel, two distinct activity levels were considered: (a)
perception of flight activity (whether the activity was detected or not) and (b) flight
activity (bee counts when activity was detected). Our approach provides a
multi-dimensional assessment (accuracy and precision in perception and counts) of
reliability in citizen science data for a non-model insect organism.

2. Material and Methods

During an outreach course on meliponiculture and citizen science held in July

2020, participants were invited and trained to perform and pilot-test a structured
protocol aimed at the monitoring of flight activity for Tetragonisca angustula (Latreille,
1811), a stingless bee widely distributed in Brazil and commonly reared by
beekeepers across the country. The protocol stated that citizen scientists had to film
the nest’s entrance for 30 s within different time intervals (between 7:00 a.m. and
9:00 a.m.; 11:00 a.m. and 1:00 p.m.; 3:00 p.m. and 5:00 p.m.). Data collection
consisted of watching the videos and counting how many bees entered (entrance),
how many left the nest (exit), and how many came in carrying pollen (pollen) in that
period (Figure 1A). As stingless bees carry pollen attached to the pollen baskets in
the hind legs (Figure 1B), pollen loads were visible and could be identified in video
recordings. Data submission was carried on a web system developed exclusively for
this purpose (https://beekeep.pcs.usp.br, in Portuguese, University of São Paulo,

89
São Paulo, Brazil—access date: 23 August 2021), also collecting other relevant
variables for further studies.

Figure 1. (A) A frame of one of the received videos showing a bee approaching the nest entrance
tube and some guard bees. (B) An image of a bee carrying pollen attached to its hind legs. Photo by
André Matos.

Of more than 400 submitted videos, 42 were randomly selected for this study
purposes, along with the counts (Supplementary Material 1) provided by citizen
scientists at the time of video submission (these participants are hereafter called “cs
original”). The videos were divided into seven groups of six videos each. For the
replication analyses among citizen scientists, each group of videos was assessed by
at least 11 participants from a total set of 101 citizen scientists (from now on called
“cs replicators”), none of them included in the cs original group. Group size varied
from 11 to 19, none of them analyzed videos of more than one group. For the
validation process, a set of five experts in stingless bee behavior analyzed all videos,
including three authors of this study. Thus, at the end of the quality control process,
there were 2574 countings (considering 858 countings both on entrances, exits, and
pollen), 126 of which were from the cs original (42 individuals × 1 video per individual
× 3 countings per video), 1818 from the cs replicators (101 individuals × 6 videos per
individual × 3 countings per video), and 630 from experts (5 individuals × 42 videos
per individual × 3 countings per video) (Figure 2).

90
 Figure 2. Distribution of the 42 videos for each group: cs original (n = 42 individuals—1:1 video), cs
replicator (n = 101 individuals—1:6 videos) and experts (n = 5 individuals—1:42 videos).

Data Analysis

The groups (cs original, cs replicators, and experts) and video quality were
considered potential sources of observed differences. In order to assess video
quality, four variables were assembled through a Principal Component Analysis
(PCA): Mean Structural SIMilarity (MSSIM), Focus, Contrast, and Frames per
Second (FPS). MSSIM was obtained using a custom script that relies on OpenCV
4.5.1 implementation of MSSIM as described by Wang et al. [37]. We also
considered other traditional metrics such as MSE (Mean Square Error) and the
correlated PSNR (Peak Signal-to-Noise Ratio), but MSSIM is a better metric when
considering the human perception of the video quality [38]. Focus and contrast
indicate qualitatively if the nest was in the foreground and if it was possible to
differentiate the bees from the background, respectively. These two metrics were
determined by a designer specialist, who watched the videos and checked both
attributes in each. FPS indicated the number of frames per second and was
extracted from video metadata or inferred using the file size and the video bitrate

91
(uncompressed) when the metadata was missing or wrong (e.g., recorder used
variable bitrate). Before performing the PCA, FPS data were scaled by subtracting
each value from the mean and dividing it by the standard deviation. PCA Axis 1
explained 68% of the data variability and was used as a proxy for video quality in our
analyses.

Generalized Linear Mixed-Effects Models (GLMER) were adjusted to analyze

accuracy for replication and validation processes. A Boolean variable was created to
indicate the presence (when greater than zero) or absence of activity and to assess
the probability of flight activity perception. The perception of flight activity was
modeled as a binary response (presence—when the participant responded with a
value greater than zero for the specific activity—or absence of activity), following a
Bernoulli distribution. At the same time, the effective counts of activity were modeled
using a Poisson distribution (using only data for which counts were greater than
zero). The group was included in the model as a fixed effect (cs original compared to
cs replicators for replication analyses; cs replicators compared to experts for
validation analyses). The video quality was included in the models as a covariate. A
random effect was set for the videos to account for dependencies in the data, with
participant identity nested within each video. Initially, complete models with response
variables for each bee activity were set (entrances, exits, and pollen). Then, reduced
models were adjusted by removing the fixed effects, one by one, until the null model.
Likelihood-ratio tests were employed to compare models and to select the best
model in each analysis. Overdispersion was assessed by verifying the data’s
standard deviation (sd) against the sd of simulated data. The estimated coefficients
were back-log-transformed using the exponential function to obtain the values of
odds or odds ratio from the models.

The Median Absolute Deviation (MAD) was used as an indicator of counting

dispersion for both the group of cs replicators and the group of experts. All the
counts were considered in this analysis, even when no activity was found (zero
values). Data from cs original could not be included in this analysis, as no replicates
were performed, and hence no variation could be measured. Like other studies that
analyzed count-based protocols, we chose the median-based metric because data
were non-parametric, and the median minimizes the influence of some extreme

92
countings [16,39]. The mean of the MAD of each video was calculated for each
group and compared through a paired Wilcoxon signed-rank test to verify possible
statistically significant differences between the groups. This non-parametric test was
used because the paired differences in MAD values between groups were not
approximately normally distributed. We also analyzed the correlation between the
mean of MAD and the median of the activities counts to assess whether there was a
relationship between bee activity rates and variation in counts in our protocol, using
the Pearson correlation coefficient.

All analyses were performed in R (version 4.0.4) [40], employing the functions
and respective packages: scale (base), prcomp (stats), GLMER (lme4), anova
(stats), mad (stats), wilcox.test (stats), cor.test (stats) and testDispersion (DHARMa).

3. Results

3.1. Perception and Effective Countings

The perception of entrance activity from the videos ranged from 69% (cs original
and experts) to 70% (cs replicators), while the perception of exit activity ranged from
76% (experts) to 79% (cs replicators). Perception of pollen was much lower than the
previous activities and varied from 14% (cs original) to 34% (cs replicators) (Figure
3). For those videos with perceived activity, the median for entrance counts was 5 for
all the groups; medians for exit counts were 6, 5, and 6, while medians for pollen
counts were 1, 2, and 1 (for cs original, cs replicators, and experts, respectively)
(Figure 4).

93
Figure 3. Absolute frequency of perception of activity (entrance, exit, pollen) by the groups (cs
original—in blue —, cs replicator—in red—, expert—in yellow) (see also Figure 2).

Figure 4. Medians and dispersion of effective countings for: (A) entrances; (B) exits; (C) pollen,
performed by cs original (in blue), cs replicators (in red), and experts (in yellow). The dots are outliers,
countings over or under 1.5 times the interquartile range.

3.2. Accuracy

Regarding replication analyses (comparison between cs original and cs

replicator), the likelihood-ratio tests indicated the null models as the best ones for the
perception of bees entering and leaving the nest (Table 1). However, for pollen
perception, group effect was found. In this case, cs replicators presented odds
increased by a factor of 4.63 compared to cs original (Table 2). The null models
were the best also for the counts of entrance, exit, and pollen (Table 1). These
results indicate that both groups performed countings quite similarly. Video quality
did not present any effect in these analyses.

94
 Table 1. Likelihood-ratio tests for model selection in the replication (comparison between cs
original and cs replicator) and validation (comparison between cs replicators and experts) analyses.
χ2 statistic, with the respective degrees of freedom (Df), and p-value for each test, are presented.
Significant differences between models are followed by an asterisk.

Fixed Effect
Response Starting Model Removed χ2 Df p-Value
group + video
quality group 156 1 693
entrance video quality video quality 45 1 832
group + video
quality group 356 1 551
exit video quality video quality 65 1 799
group + video
quality group 10.852 1 0.001 *
group + video
quality video quality 64 1 801
Perception pollen group group 10.857 1 0.001 *
group + video
quality group 1.674 1 196
entrance video quality video quality 3 1 957
group + video
quality group 658 1 417
exit video quality video quality 1 1 981
group + video
quality group 1.367 1 242
Replication Count pollen video quality video quality 63 1 802
group + video
quality group 516 1 472
entrance video quality video quality 56 1 812
group + video
quality group 592 1 442
exit video quality video quality 3 1 958
group + video
quality group 22.325 1 0.001 *
group + video
quality video quality 77 1 781
Perception pollen group group 22.330 1 0.001 *
group + video
quality group 39 1 843
entrance video quality video quality 38 1 845
group + video
quality group 35 1 851
exit video quality video quality 1 1 981
group + video
quality group 315 1 575
Validation Count pollen video quality video quality 219 1 640

95
Table 2. Parameter estimates of the final models for pollen perception in replication and validation
analyses.

Model Comparison Predictor Estimate SE Pr(>|z|) Odds/Odds Ratio

cs original (intercept) −2.7561 333 <0.001 0.06
Replication cs replicators 1.5332 508 3 4.63
experts (intercept) −2.3004 344 <0.001 0.1
Pollen
perception Validation cs replicators 1.0529 229 <0.001 2.87

In the validation analyses (comparison between cs replicators and experts), null

models were the best ones for the perception of entrance and exit and for the counts
of entrance, exit, and pollen (Table 1). However, for pollen perception, the best
model included the group variable, where the cs replicators had odds of perceiving
pollen increased by a factor of 2.87 when compared to experts (Table 2). Here, the
video quality index also did not add relevant information to explain the participants’
data. Even though the model for pollen counts presented significant overdispersion
despite adjustments (Supplementary Material Table S1), i.e., the residual variance
was larger than expected under the fitted model, no effect of the tested variables
was found.

3.3. Precision

The MAD means were statistically equal between cs replicators and experts for
entrance (V = 24.5, p-value 0.071) and exit (V = 54, p-value 0.052) counts. However,
the MAD mean was higher for pollen counts for cs replicators in relation to experts (V
= 85, p-value 0.005), who exhibited low dispersion for this task (Figure 5 and
Supplementary Material Figure S1). These results are in accordance with the
accuracy analyses, which presented significant differences for pollen perception
between groups.

96
Figure 5. Mean of Median Absolute Deviation (MAD) of all videos for entrance, exit, and pollen
counts, for cs replicators (in red) and experts (in yellow). Vertical bars indicate the Confidence
Intervals (95%).

Data dispersion for the countings was positively correlated with the amount of
bees in activity in the videos: entrance, cs replicators (r = 0.8, p-value < 0.001) and
experts (r = 0.78, p-value < 0.001); exit, cs replicators (r = 0.57, p-value < 0.001) and
experts (r = 0.46, p-value 0.002). For entrance and exit, the pattern of increasing
dispersion is similar for both groups, however, for pollen, the correlation is stronger
for citizen scientists: cs replicators (r = 0.94, p-value < 0.001) and experts (r = 0.43,
p-value 0.004) (Figure 6).

Figure 6. Correlation between the median of counts and dispersion (MAD) values of each video by
group (cs replicator in red and experts in yellow), with tendency line (linear model line). (A) entrances;
(B) exits; (C) pollen.

97
4. Discussion

Count-based activities are tasks with low to medium skill or training requirements
[8]. In the present study, our results evidenced that, depending on the task, the
accuracy of different groups can be affected. This was especially true when the
perception of bees carrying pollen was considered. Thus, specific ability levels may
be necessary to guarantee reliability in distinct contexts. For instance, perceiving a
moving bee is significantly different from perceiving pollen present at a bee
corbicula, which is a much smaller target and may require more volunteer training to
reach the same level of quality of the experts. In the work of Bieluch et al. [41], CS
program coordinators were interviewed about the contribution of volunteers in fish
count-based protocols. They highlighted some aspects that can influence the
counting accuracy, like the poor weather, high numbers of individuals passing at
once, and the physical attributes of the counting site. Here, attributes of the video,
which could be a proxy for context variation, did not affect countings. Target size may
also influence perception, and large individuals and distinctive shapes can help to
produce more accurate data [39]. It could pose as a barrier for data quality in our
study since the size of the studied bees is small, varying between 4 mm and 5 mm,
imposing some difficulty for those who are analyzing the videos [42]. Finally,
stingless bees normally speed up when near the nest entrance, which could also
hinder the perception of bees carrying pollen by citizen scientists [43].

The observed divergence in pollen perception by cs replicators compared to the

other groups cannot be fully explained in this study, where experts presented
conservative counts, while cs replicators exhibited a permissive performance. We
hypothesized that due to citizen scientists’ eagerness to find some pollen and their
lack of experience, they could overestimate the perception of this activity. If this is
true, with more practical training effort in this protocol, we expect that cs replicators
would present lower and less variation in the perception of pollen loads [44]. Some
questionable research practices, such as falsification (wilful or unintended distortion
of data or results), can negatively impact research [45], so they must be seriously
considered when resulting from citizen science practices.

98
 Generally, false negatives are a concern for researchers that deal with
biodiversity occurrence data from citizen science (e.g., [46,47], but false positives
are generally considered unimportant, although they can lead to severe biases in
conclusions about ecological systems [48]. Overestimation and underestimation of
counts in citizen science protocols are reported in the literature. For instance, citizen
scientists underestimated experts’ countings of seals and sea lions in static images
[39], countings of stomata and epidermal cells in static images [49], and the
estimation of caterpillar density [50]. On the other hand, they overestimated fly and
beetle density when applying a visual survey protocol in natural habitats [50]. These
results varied according to the protocol and factors related to the individuals, such as
experience and training [51]. However, empirical evidence suggests that citizen
science data quality has often been sufficient for the projects’ aims, and differences
between citizen scientists’ and professionals’ data were not significant in most cases,
e.g., 61.6% of studies showing no significant differences between citizen scientists
and professionals in Aceves-Bueno et al. [27].

Considering the count precision in our study, intra-group dispersion was found in
both groups, which is naturally expected in science, regardless of the individuals’
expertise. For instance, in Swanson et al. [16], during an expert verification, precise
counts of specimens were unresolvable in many of the images they were analyzing,
and the specialists agreed on the number of individuals only 74% of the time. The
authors concluded that multiple citizen scientists classifying an image could be more
reliable if compared to a single person, even if this person is an expert. In our study,
when more bees were in flight activity, more dispersion was found in all groups,
which can reveal a greater difficulty for counting many bees at the same time. Other
confounding factors could inflate countings, such as the presence of guards at the
nest entrance. These guards are larger and heavier than the regular workers,
hovering or standing next to the nest entrance tube (Figure 1A) [52], possibly being
confused with bees in foraging activity when activity is high. We also found that CS
data, specifically for pollen counts, showed higher dispersion than expert data.
Likewise, data variability among citizen scientists was tested in Fehri et al. [53], in
which the volunteers were engaged and trained on using rain gauge tools. Data
dispersion was slightly higher in specific situations (high rainfall events), but in

99
contrast, other events measurements (lower precipitation) showed more consistency
for the group composed of citizen scientists. More straightforward tasks performed
by volunteers tend to present less data dispersion when compared to more complex
ones [53,54]. Additionally, volunteers generally improve their accuracy as they gain
experience within a project [8]. The continuous execution of a task can promote
personal learning and progress of the required skill [44] and improve data quality
[27]. In our citizen science initiative, the participants of the outreach course learned
to perform the protocol based on three video lessons and online guidance since any
practical activity was not possible due to the restrictions imposed by the COVID-19
pandemic. Therefore, it is possible that an increased effort in the training of
volunteers could lead to higher quality data in the present study, as has already been
reported for other citizen science studies (e.g., [19,55,56]). Clear and objective
protocols can also help in the rigorous collection of data [23,57].

Here, replication was used as a way to test for data quality, which is a practice
reported to produce high rates of accuracy [16,39,58], since the combination of
different and independent contributions decreases the errors observed individually in
each one, according to the “Wisdom of Crowds” principle [59]. This quality assurance
procedure can be used by default in the protocol design by proposing that all the
collected data should be confirmed by a set of volunteers, or only in cases of
unusual records, for example. A challenge with this approach is to define the value of
agreement between individuals [39], sometimes with the support of expert checks.
The level of accuracy needed will likely depend on the research question and the
ability to perform post hoc statistical manipulation on these data [54]. In our case,
replication provided reliable CS data for the counts of bees entering or leaving the
nest. However, the perception of pollen was consistently different between groups
(Figure 3, Table 1 and Table 2) and countings were also highly variable for citizen
scientists (Figure 5 and Figure 6). Thus, although effective counts of pollen were
reliable, both in replication and validation analyses (Figure 4), the observed variance
in the replicators’ data can impose some data quality issues and compromise the
application of these specific data.

100
 As a consequence of these results obtained for pollen in the present pilot test of
the protocol, a checkbox option labeled “pollen count was performed” was included
in the platform, making it possible for citizen scientists to report an actual absence of
pollen activity (“real” zeros) differently from the zeros that represent that they were
unable to perform the task. Indeed, including an “I don’t know” option was shown to
enhance data quality and contribute to the agreement among participants in a citizen
science initiative [60]. In addition, we realized the importance of slowing down the
video speed to make pollen loads easier to see. Thus we have also included this
functionality in our platform as well. Future works may include developing an
agreement algorithm to determine the consensual entrance, exit, and pollen activity
in each video and the use of artificial intelligence to perform automatic counts.

5. Conclusions

Our results indicate that the flight activity protocol for stingless bees provides
reliable data for bees entering and leaving the nest since original, replicated, and
expert data were similar. These results are in accordance with our initial hypotheses
that CS data do not differ statistically from those provided by specialists. However, a
significant difference was found for the perception of pollen loads, with the cs
replicator diverging from the experts and the cs original. Despite that, for those
videos in which there was an agreement about pollen presence, i.e., in which both
groups identified bees carrying pollen, the effective counts were very similar and
confirmed the reliability of CS-collected data.

Data quality is a common concern in the era of data, and mechanisms to

evaluate and improve quality are essential to ensure data applicability in its intended
purpose. In CS, the problems are, in fact, comparable to those found in the
traditional scientific exercise [3,8] and both models need to implement specific
measures to guarantee the data quality and the other obtained products [54,61]. In
CS, these quality requirements can be research questions of interest to participants,
viable protocols, consistent training, evidence of observations, replication, expert
review, among others. Several studies in different fields of knowledge, including the
present one, report positive results and highlight characteristics that justify the

101
investment in volunteers to act as scientists [12,27,62,63]. Depending on the
particularities of the required task, adequate training and specific skill levels may be
necessary, as in any other scientific approach. Our study highlights the importance of
a multi-dimensional approach in CS data quality assessments to identify potential
pitfalls and adequate protocols to improve data collection and use of CS data in
research.

Supplementary Materials

The following are available online at

https://www.mdpi.com/article/10.3390/insects12090766/s1, Supplementary
Material 1: Excel file with complete data-set, containing citizen scientists original
(cs-original tab), replicated (cs-rep tab), and expert (exp tab) counts, along with
video quality metric; (vq tab); Supplementary Material 2: Table S1: Dispersion of
residual variance for each model. Significant dispersion models are highlighted in
bold, Figure S1: Median Absolute Deviation (MAD) value of each video for replicator
citizen scientists and experts in different bee activities.

Author Contributions

Conceptualization, J.N.L., N.P.G.-L., S.K., C.B., T.M.F. and B.A.; methodology,

J.N.L., N.P.G.-L., S.K., C.B. and B.A.; validation, J.N.L., N.P.G.-L., S.K. and C.B.;
formal analysis, J.N.L., N.P.G.-L. and S.K.; investigation, J.N.L., N.P.G.-L., S.K. and
C.B.; data curation, J.N.L., N.P.G.-L. and S.K.; writing—original draft preparation,
J.N.L., N.P.G.-L., S.K., C.B., T.M.F. and B.A.; writing—review and editing, J.N.L.,
N.P.G.-L., S.K., C.B., T.M.F., B.A. and A.M.S.; visualization, J.N.L., N.P.G.-L. and
S.K.; supervision, N.P.G.-L., S.K., T.M.F. and A.M.S.; project administration,
N.P.G.-L., B.A., T.M.F. and A.M.S.; funding acquisition, A.M.S. All authors have read
and agreed to the published version of the manuscript.

102
Funding

This research was funded by Fundação de Amparo e Apoio à Pesquisa do

Estado de São Paulo (FAPESP, grant numbers 2018/14994-1 and 2019/26760-8).
This study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal
de Nível Superior - Brazil (CAPES) - Finance Code 001; J.N.L grant number
88882.333367/2019-01; C.B. grant number 88882.377160/2019-01 and also by
Conselho Nacional de Desenvolvimento Científico e Tecnológico - Brazil (CNPq)
A.M.S grant number 312.605/2018-8.

Institutional Review Board Statement

Ethical review and approval were waived for this study, as all analyzes were
performed on counts that citizen scientists and experts performed on the videos. No
collection of personal information or evaluation of personal data (e.g., insights or
learning outcomes) was carried out with the citizen scientists. No handling or
collection of bees was carried out.

Informed Consent Statement

Patient consent was waive as no collection of personal information or evaluation

of personal data was carried out with the citizen scientists.

Data Availability Statement

The raw data used in this study are provided as Supplementary Material.

103
Acknowledgments

We thank all citizen scientists that participated in this study providing the videos
and counts; the Provost of Culture and Extension at USP for the support to the
outreach course on meliponiculture and citizen science; the experts Patrícia
Nunes-Silva and Sergio Dias Hilário for their contribution to the validation stage, and
Paula Marques Drago Oliveira for contributing to video quality index. We also would
like to acknowledge the SURPASS2 project (Safeguarding pollination services in a
changing world), FAPESP, and CAPES for their support.

Conflicts of Interest

The authors declare no conflict of interest.

Abbreviations

The following abbreviations are used in this manuscript:

CS Citizen Science
cs Citizen Scientist
Df Degrees of Freedom
DQ Data Quality
FPS Frames per Second
GLMER Generalized Linear Mixed-Effects Model
MAD Median Absolute Deviation
MSE Mean Square Error
MSSIM Mean Structural SIMilarity
PCA Principal Component Analysis
PSNR Peak Signal-to-Noise Ratio
QA Quality Assurance
QC Quality Control
sd Standard Deviation
SE Standard Error

104
References

1. Bonney, R.; Ballard, H.; Jordan, R.; McCallie, E.; Phillips, T.; Shirk, J.;
Wilderman, C.C. Public Participation in Scientific Research: Defining the Field
and Assessing Its Potential for Informal Science Education; A CAISE Inquiry
Group Report. Online Submission; Center for Advancement of Informal
Science Education (CAISE): Washington, DC, USA, 2009. [Google Scholar]

2. Wiggins, A.; Crowston, K. From Conservation to Crowdsourcing: A Typology

of Citizen Science. In Proceedings of the 2011 44th Hawaii International
Conference on System Sciences, Kauai, HI, USA, 4–7 January 2011; pp.
1–10. [Google Scholar] [CrossRef][Green Version]

3. Theobald, E.J.; Ettinger, A.K.; Burgess, H.K.; DeBey, L.B.; Schmidt, N.R.;
Froehlich, H.E.; Wagner, C.; HilleRisLambers, J.; Tewksbury, J.; Harsch, M.A.;
et al. Global change and local solutions: Tapping the unrealized potential of
citizen science for biodiversity research. Biol. Conserv. 2015, 181, 236–244.
[Google Scholar] [CrossRef][Green Version]

4. Chandler, M.; See, L.; Copas, K.; Bonde, A.M.; López, B.C.; Danielsen, F.;
Legind, J.K.; Masinde, S.; Miller-Rushing, A.J.; Newman, G.; et al.
Contribution of citizen science towards international biodiversity monitoring.
Biol. Conserv. 2017, 213, 280–294. [Google Scholar] [CrossRef][Green
Version]

5. Fritz, S.; See, L.; Carlson, T.; Haklay, M.M.; Oliver, J.L.; Fraisl, D.; Mondardini,
R.; Brocklehurst, M.; Shanley, L.A.; Schade, S.; et al. Citizen science and the
United Nations sustainable development goals. Nat. Sustain. 2019, 2,
922–930. [Google Scholar] [CrossRef]

6. Pipino, L.L.; Lee, Y.W.; Wang, R.Y. Data Quality Assessment. Commun. ACM
2002, 45, 211–218. [Google Scholar] [CrossRef]

7. United States Environmental Protection Agency. Guidance for Data Quality

Assessment: Practical Methods for Data Analysis EPA QA/G-9 (QA00
Update); United States Environmental Protection Agency: Washington, DC,
USA, 2000.

105
8. Kosmala, M.; Wiggins, A.; Swanson, A.; Simmons, B. Assessing data quality
in citizen science. Front. Ecol. Environ. 2016, 14, 551–560. [Google Scholar]
[CrossRef][Green Version]

9. Ottinger, G. Buckets of resistance: Standards and the effectiveness of citizen

science. Sci. Technol. Hum. Values 2010, 35, 244–270. [Google Scholar]
[CrossRef]

10. Zhang, G.; Zhu, A.X. The representativeness and spatial bias of volunteered
geographic information: A review. Ann. GIS 2018, 24, 151–162. [Google
Scholar] [CrossRef]

11. Fowler, A.; Whyatt, J.D.; Davies, G.; Ellis, R. How reliable are citizen-derived
scientific data? Assessing the quality of contrail observations made by the
general public. Trans. GIS 2013, 17, 488–506. [Google Scholar] [CrossRef]

12. Lewandowski, E.; Specht, H. Influence of volunteer and project characteristics

on data quality of biological surveys. Conserv. Biol. 2015, 29, 713–723.
[Google Scholar] [CrossRef]

13. Callaghan, C.; Lyons, M.; Martin, J.; Major, R.; Kingsford, R. Assessing the
reliability of avian biodiversity measures of urban greenspaces using eBird
citizen science data. Avian Conserv. Ecol. 2017, 12, 12. [Google Scholar]
[CrossRef][Green Version]

14. Steinke, J.; van Etten, J.; Zelan, P.M. The accuracy of farmer-generated data
in an agricultural citizen science methodology. Agron. Sustain. Dev. 2017, 37,
32. [Google Scholar] [CrossRef][Green Version]

15. Gardiner, M.M.; Allee, L.L.; Brown, P.M.; Losey, J.E.; Roy, H.E.; Smyth, R.R.
Lessons from lady beetles: Accuracy of monitoring data from US and UK
citizen-science programs. Front. Ecol. Environ. 2012, 10, 471–476. [Google
Scholar] [CrossRef][Green Version]

16. Swanson, A.; Kosmala, M.; Lintott, C.; Packer, C. A generalized approach for
producing, quantifying, and validating citizen science data from wildlife
images. Conserv. Biol. 2016, 30, 520–531. [Google Scholar] [CrossRef]

106
17. Fuccillo, K.K.; Crimmins, T.M.; de Rivera, C.E.; Elder, T.S. Assessing
accuracy in citizen science-based plant phenology monitoring. Int. J.
Biometeorol. 2015, 59, 917–926. [Google Scholar] [CrossRef] [PubMed]

18. Ratnieks, F.L.; Schrell, F.; Sheppard, R.C.; Brown, E.; Bristow, O.E.;
Garbuzov, M. Data reliability in citizen science: Learning curve and the effects
of training method, volunteer background and experience on identification
accuracy of insects visiting ivy flowers. Methods Ecol. Evol. 2016, 7,
1226–1235. [Google Scholar] [CrossRef]

19. van der Velde, T.; Milton, D.A.; Lawson, T.; Wilcox, C.; Lansdell, M.; Davis, G.;
Perkins, G.; Hardesty, B.D. Comparison of marine debris data collected by
researchers and citizen scientists: Is citizen science data worth the effort?
Biol. Conserv. 2017, 208, 127–138. [Google Scholar] [CrossRef]

20. Alabri, A.; Hunter, J. Enhancing the quality and trust of citizen science data. In
Proceedings of the 2010 IEEE Sixth International Conference on E-Science,
Brisbane, QLD, Australia, 7–10 December 2010; pp. 81–88. [Google
Scholar]

21. Yu, J.; Wong, W.K.; Hutchinson, R.A. Modeling experts and novices in citizen
science data for species distribution modeling. In Proceedings of the 2010
IEEE International Conference on Data Mining, Sydney, NSW, Australia,
13–17 December 2010; pp. 1157–1162. [Google Scholar]

22. Tweddle, J.C.; Robinson, L.D.; Pocock, M.; Roy, H.E. Guide to Citizen
Science: Developing, Implementing and Evaluating Citizen Science to Study
Biodiversity and the Environment in the UK; NERC/Centre for Ecology &
Hydrology: Wallingford, UK, 2012. [Google Scholar]

23. Freitag, A.; Meyer, R.; Whiteman, L. Strategies employed by citizen science
programs to increase the credibility of their data. Citiz. Sci. Theory Pract.
2016, 1, 2. [Google Scholar] [CrossRef][Green Version]

24. Bonter, D.N.; Cooper, C.B. Data validation in citizen science: A case study
from Project FeederWatch. Front. Ecol. Environ. 2012, 10, 305–307. [Google
Scholar] [CrossRef]

107
25. Follett, R.; Strezov, V. An analysis of citizen science based research: Usage
and publication patterns. PLoS ONE 2015, 10, e0143687. [Google Scholar]
[CrossRef] [PubMed][Green Version]

26. Kelling, S.; Johnston, A.; Hochachka, W.M.; Iliff, M.; Fink, D.; Gerbracht, J.;
Lagoze, C.; La Sorte, F.A.; Moore, T.; Wiggins, A.; et al. Can observation skills
of citizen scientists be estimated using species accumulation curves? PLoS
ONE 2015, 10, e0139600. [Google Scholar] [CrossRef] [PubMed][Green
Version]

27. Aceves-Bueno, E.; Adeleye, A.M.S.; Feraud, M.; Huang, Y.; Tao, M.; Yang, Y.;
Anderson, S.E. The accuracy of citizen science data: A quantitative review.
Bull. Ecol. Soc. Am. 2017, 98, 278–290. [Google Scholar] [CrossRef][Green
Version]

28. Koffler, S.; Barbiéri, C.; Ghilardi-Lopes, N.P.; Leocadio, J.N.; Albertini, B.;
Francoy, T.M.; Saraiva, A.M. A buzz for sustainability and conservation: The
growing potential of citizen science studies on bees. Sustainability 2021, 13,
959. [Google Scholar] [CrossRef]

29. Wolowski, M.; Agostini, K.; Rech, A.; Varassin, I.; Maués, M.; Freitas, L.;
Carneiro, L.; BUENO, R.d.O.; Consolaro, H.; Carvalheiro, L.; et al. Relatório
Temático Sobre Polinização, Polinizadores e Produção de Alimentos No
Brasil; Editora Cubo: São Carlos, Brazil, 2019. [Google Scholar]

30. Cortopassi-Laurino, M.; Imperatriz-Fonseca, V.L.; Roubik, D.W.; Dollin, A.;

Heard, T.; Aguilar, I.; Venturieri, G.C.; Eardley, C.; Nogueira-Neto, P. Global
meliponiculture: Challenges and opportunities. Apidologie 2006, 37, 275–292.
[Google Scholar] [CrossRef][Green Version]

31. Jaffé, R.; Pope, N.; Carvalho, A.T.; Maia, U.M.; Blochtein, B.; de Carvalho,
C.A.L.; Carvalho-Zilse, G.A.; Freitas, B.M.; Menezes, C.; de Fátima Ribeiro,
M.; et al. Bees for development: Brazilian survey reveals how to optimize
stingless beekeeping. PLoS ONE 2015, 10, e0121157. [Google Scholar]
[CrossRef] [PubMed]

32. Ascher, J.S.; Pickering, J. Discover Life Bee Species Guide and World
Checklist (Hymenoptera: Apoidea: Anthophila). Available online:

108
 http://www.discoverlife.org/mp/20q?guide=Apoidea_species (accessed
on 24 August 2021).

33. Lemelin, R.H. Entomotourism and the stingless bees of Mexico. J. Ecotourism
2020, 19, 168–175. [Google Scholar] [CrossRef]

34. Barbiéri, C.; Francoy, T.M. Theoretical model for interdisciplinary analysis of
human activities: Meliponiculture as an activity that promotes sustainability.
Ambiente Soc. 2020, 23, 1–19. [Google Scholar] [CrossRef]

35. Gray, A.; Adjlane, N.; Arab, A.; Ballis, A.; Brusbardis, V.; Charrière, J.D.;
Chlebo, R.; Coffey, M.F.; Cornelissen, B.; Amaro da Costa, C.; et al. Honey
bee colony winter loss rates for 35 countries participating in the COLOSS
survey for winter 2018–2019, and the effects of a new queen on the risk of
colony winter loss. J. Apic. Res. 2020, 59, 744–751. [Google Scholar]
[CrossRef]

36. Gary, N.E. A method for evaluating honey bee flight activity at the hive
entrance. J. Econ. Entomol. 1967, 60, 102–105. [Google Scholar]
[CrossRef]

37. Wang, Z.; Bovik, A.C.; Sheikh, H.R.; Simoncelli, E.P. Image quality
assessment: From error visibility to structural similarity. IEEE Trans. Image
Process. 2004, 13, 600–612. [Google Scholar] [CrossRef][Green Version]

38. Lin, D.C.C.; Chau, P.M. Objective human visual system based video quality
assessment metric for low bit-rate video communication systems. In
Proceedings of the 2006 IEEE Workshop on Multimedia Signal Processing,
Victoria, BC, Canada, 3–6 October 2006; pp. 320–323. [Google Scholar]

39. Wood, S.A.; Robinson, P.W.; Costa, D.P.; Beltran, R.S. Accuracy and
precision of citizen scientist animal counts from drone imagery. PLoS ONE
2021, 16, e0244040. [Google Scholar] [CrossRef]

40. R Core Team. R: A Language and Environment for Statistical Computing; R

Foundation for Statistical Computing: Vienna, Austria, 2021. [Google
Scholar]

109
41. Bieluch, K.H.; Willis, T.; Smith, J.; Wilson, K.A. The complexities of counting
fish: Engaging citizen scientists in fish monitoring. Maine Policy Rev. 2017,
26, 9–18. [Google Scholar]

42. Wittmann, D. Aerial defense of the nest by workers of the stingless bee
Trigona (Tetragonisca) angustula (Latreille) (Hymenoptera: Apidae). Behav.
Ecol. Sociobiol. 1985, 16, 111–114. [Google Scholar] [CrossRef]

43. Tichit, P.; Alves-dos Santos, I.; Dacke, M.; Baird, E. Accelerated landing in a
stingless bee and its unexpected benefits for traffic congestion. Proc. R. Soc.
B 2020, 287, 20192720. [Google Scholar] [CrossRef] [PubMed][Green
Version]

44. Schmeller, D.S.; HENRY, P.Y.; Julliard, R.; Gruber, B.; Clobert, J.; Dziock, F.;
Lengyel, S.; Nowicki, P.; Deri, E.; Budrys, E.; et al. Advantages of
volunteer-based biodiversity monitoring in Europe. Conserv. Biol. 2009, 23,
307–316. [Google Scholar] [CrossRef] [PubMed]

45. Fanelli, D. How many scientists fabricate and falsify research? A systematic
review and meta-analysis of survey data. PLoS ONE 2009, 4, e5738. [Google
Scholar] [CrossRef][Green Version]

46. Sewell, D.; Beebee, T.J.; Griffiths, R.A. Optimising biodiversity assessments
by volunteers: The application of occupancy modelling to large-scale
amphibian surveys. Biol. Conserv. 2010, 143, 2102–2110. [Google Scholar]
[CrossRef]

47. Hochachka, W.M.; Fink, D.; Hutchinson, R.A.; Sheldon, D.; Wong, W.K.;
Kelling, S. Data-intensive science applied to broad-scale citizen science.
Trends Ecol. Evol. 2012, 27, 130–137. [Google Scholar] [CrossRef]

48. Miller, D.A.; Weir, L.A.; McClintock, B.T.; Grant, E.H.C.; Bailey, L.L.; Simons,
T.R. Experimental investigation of false positive errors in auditory species
occurrence surveys. Ecol. Appl. 2012, 22, 1665–1674. [Google Scholar]
[CrossRef] [PubMed]

49. Soul, L.C.; Barclay, R.S.; Bolton, A.; Wing, S.L. Fossil Atmospheres: A case
study of citizen science in question-driven palaeontological research. Philos.
Trans. R. Soc. B 2019, 374, 20170388. [Google Scholar] [CrossRef][Green
Version]

110
50. Hurlbert, A.H.; Hayes, T.E.; McKinnon, T.N.; Goforth, C.L. Caterpillars Count!
A citizen science project for monitoring foliage arthropod abundance and
phenology. Citiz. Sci. Theory Pract. 2019, 4, 1–12. [Google Scholar]
[CrossRef][Green Version]

51. Elliott, K.C.; Rosenberg, J. Philosophical foundations for citizen science. Citiz.
Sci. Theory Pract. 2019, 4, 9. [Google Scholar] [CrossRef][Green Version]

52. Grüter, C.; Menezes, C.; Imperatriz-Fonseca, V.L.; Ratnieks, F.L. A

morphologically specialized soldier caste improves colony defense in a
neotropical eusocial bee. Proc. Natl. Acad. Sci. USA 2012, 109, 1182–1186.
[Google Scholar] [CrossRef][Green Version]

53. Fehri, R.; Khlifi, S.; Vanclooster, M. Testing a citizen science water monitoring
approach in Tunisia. Environ. Sci. Policy 2020, 104, 67–72. [Google Scholar]
[CrossRef]

54. Crall, A.W.; Newman, G.J.; Stohlgren, T.J.; Holfelder, K.A.; Graham, J.; Waller,
D.M. Assessing citizen science data quality: An invasive species case study.
Conserv. Lett. 2011, 4, 433–442. [Google Scholar] [CrossRef]

55. Foster-Smith, J.; Evans, S.M. The value of marine ecological data collected
by volunteers. Biol. Conserv. 2003, 113, 199–213. [Google Scholar]
[CrossRef]

56. Hunter, J.; Alabri, A.; van Ingen, C. Assessing the quality and trustworthiness
of citizen science data. Concurr. Comput. Pract. Exp. 2013, 25, 454–466.
[Google Scholar] [CrossRef]

57. Geoghegan, H.; Dyke, A.; Pateman, R.; West, S.; Everett, G. Understanding
Motivations for Citizen Science; Final Report on Behalf of UKEOF, University
of Reading, Stockholm Environment Institute (University of York) and
University of the West of England; UKEOF: Wiltshire, UK, 2016. [Google
Scholar]

58. Bain, R. Citizen science and statistics: Playing a part. Significance 2016, 13,
16–21. [Google Scholar] [CrossRef][Green Version]

59. Surowiecki, J. The Wisdom of Crowds; Anchor: New York, NY, USA, 2005.
[Google Scholar]

111
60. Torre, M.; Nakayama, S.; Tolbert, T.J.; Porfiri, M. Producing knowledge by
admitting ignorance: Enhancing data quality through an “I don’t know” option
in citizen science. PLoS ONE 2019, 14, e0211907. [Google Scholar]
[CrossRef]

61. Bird, T.J.; Bates, A.E.; Lefcheck, J.S.; Hill, N.A.; Thomson, R.J.; Edgar, G.J.;
Stuart-Smith, R.D.; Wotherspoon, S.; Krkosek, M.; Stuart-Smith, J.F.; et al.
Statistical solutions for error and bias in global citizen science datasets. Biol.
Conserv. 2014, 173, 144–154. [Google Scholar] [CrossRef][Green Version]

62. Silvertown, J. A new dawn for citizen science. Trends Ecol. Evol. 2009, 24,
467–471. [Google Scholar] [CrossRef]

63. Krabbenhoft, C.A.; Kashian, D.R. Citizen science data are a reliable
complement to quantitative ecological assessments in urban rivers. Ecol.
Indic. 2020, 116, 106476. [Google Scholar] [CrossRef]

112
7 - CAPÍTULO V “Evaluation of a Bee-focused Citizen Science Project:
Influences of participants’ profile in their learning”

Celso Barbiéri 1,∗ , Sheina Koffler 2 , Jailson N. Leocadio 3 , Bruno Albertini 3 ,

Tiago M.
Francoy 1 , Antonio M. Saraiva 2,3 and Natalia P. Ghilardi-Lopes 4

1 Escola de Artes, Ciências e Humanidades, University of São Paulo, R. Arlindo

Bettio 1000, 03828-000, São Paulo-SP, Brazil;
2 Instituto de Estudos Avançados, University of São Paulo, R. Praça do Relógio 109,
05508-970, São Paulo-SP, Brazil;
3 Escola Politécnica, University of São Paulo, Av. Prof. Luciano Gualberto 158, Tv. 3,
05508-010, São Paulo-SP,Brazil;
4 Centro de Ciências Naturais e Humanas, Federal University of ABC, R. Arcturus 3,
09606-070, São Bernardo do Campo-SP, Brazil
* Correspondence: celso.barbieri@usp.br
Version May 22, 2023 submitted to Sustainability

Abstract: In citizen science, training is crucial to achieving high-quality data, besides

providing benefits to citizen scientists. An important part of citizen science projects is
their evaluation, which helps researchers understand the demands of citizen
scientists and the perfecting of the project. However, evaluating learning outcomes is
complex and usually requires multidimensional analysis. In this work, we analyze the
influence of the profile of citizen scientists on their learning, considering multiple
dimensions. Citizen scientists participated in a training course in meliponiculture and
citizen science and performed a contributory citizen science protocol. The evaluation
was based on pre and post-course questionnaires. All the analyzed groups showed
a significant increase in learning for all dimensions. Variables related to the level of
education, area of knowledge of participants’ formation, and if the participant is a
meliponist or not were more influential. Participants with no beekeeping practice
showed higher learning for some dimensions and, therefore, can be a good target
group for citizen science projects focused on stingless bees. Our results indicate that

113
even when people with lower levels of formal education have lower initial scores,
they show a higher level of learning. Citizen science initiatives focused on
meliponiculture can also increase the participants’ general knowledge and practical
skills regarding bees and stingless beekeeping. Evaluation of the influence of the
profile of volunteers can contribute to planning training strategies in other citizen
science projects, which can, ultimately, positively impact the conservation of bees.
Keywords: volunteer assessment; stingless beekeeping; learning evaluation;
learning outcomes; profile

1. Introduction
Citizen Science approaches are defined as the active engagement of the general
public in scientific activities to produce new scientific knowledge [1]. This kind of
volunteer participation in scientific tasks has centuries of history [2], although the
term Citizen Science was only coined in the 1990s and this field has its popularity
growing since then.
It is commonly expected that citizen scientists will learn about the subjects related to
the project they are participating in and improve their observational or analytical skills
[3]. In fact, Citizen Science can be valuable in improving the public understanding of
science [4]. However, evaluating citizen scientists is not the main focus of many
studies, mainly designed to collect data and answer scientific questions [5,6].
Evaluation is an important step in improving citizen science projects, both in
terms of data quality and the citizen scientists’ experiences and learning [7]. A
myriad of studies proposed to assess the different impacts of Citizen Science
projects at the participant level. Some of them evaluated pro-environmental attitudes
related to participation in citizen science initiatives [8–10], scientific knowledge
[11,12], or the comprehension of the scientific process [4,13]. The individual learning
outcomes of citizen science projects, as can be seen, are multidimensional [14].
Among the Citizen Science projects involving biodiversity, we can highlight the
bees-related projects as an example of how citizen science projects may be diverse
in subjects and outcomes. In a systematic review, Koffler et al. [15] found that most
citizen science research papers on bees have beekeeping, ecological aspects, data
quality, and volunteer assessment as the central subject. Regarding volunteer
assessment, MacPhail et al. [16], based on the results of the Bumble BeeWatch
citizen science project, highlights the importance of assessing the profile of

114
volunteers to adequate strategies for generating environmental awareness and
improving data quality. In Brazil, in a study that evaluated the quality of the data
produced by citizen scientists, which performed a protocol for monitoring stingless
bee flight activity, Leocadio et al. [17] showed that citizen scientists produced
high-quality data, but when the task was complex (counting bees with pollen on 30s
videos) the results were less precise when compared to expertly produced data. As
a consequence of these results, the authors modified the protocol to improve the
quality of the collected data. The association of citizen science approaches and the
training of meliponists can be an interesting strategy to produce data on stingless
bees and promote scientific literacy. Meliponiculture is the rational keeping of
stingless bees [18]. This activity may be considered a promoter of sustainability in
the Social, Economic, Cultural, and Ecological domains, providing multiple benefits
for the stingless beekeepers, better known as meliponists [19]. Meliponiculture has
gained popularity in the last decade, and the number of meliponists has been
increasing [20]. On the other hand, one of the meliponists’ most perceived problems
is the lack of access to general knowledge and management skills [21]. Since
educational processes can promote changes in attitudes and behaviors towards
biodiversity conservation [22,23], it is essential to create opportunities for the
promotion of training courses.
In this study, we seek to understand how the profile of the participants of a
training course in meliponiculture and citizen science influences their learning
outcomes, in multiple dimensions.

2. Materials and Methods

2.1. Training and data collection context

The 4th edition of the Outreach Course in Meliponiculture and Citizen Science
(a partnership between the University of São Paulo and the Federal University of
ABC) was held between 17th and 28th February 2022. The online course content
covered the fundamentals of bee biology and its importance to ecosystems, the
basic management of stingless bees, meliponiculture, and its products, principles of
scientific thinking, and citizen science. At the end of the course, the participants were
invited to perform the stingless bees flight activity monitoring protocol [15,17]. All
classes were pre-recorded and synchronic interaction with the participants was

115
provided through live sessions to answer questions and discuss topics of interest.
Two questionnaires were applied to assess the profile and learning of citizen
scientists. The initial questionnaire (hereafter pre-questionnaire) was applied before
the course and the second (hereafter post-questionnaire) was applied upon course
completion. Only participants who completed all the course modules had access to
the post-questionnaire.
The training for the monitoring protocol was provided through: 1) pre-recorded
videos, 2) instructions available in an ebook [15], and 3) online live sessions to
answer participants’ doubts. In addition, an online forum was available for
participants to pose questions and start discussions. All supporting materials
highlighted the theoretical context of the project (environmental factors influencing
bee flight activity), the research question being investigated, and a detailed
description of each step of the protocol. The citizen science protocol (part of a
project entitled #cidadãoasf) consisted in monitoring the flight activity of bees and
submitting the data to an online platform (https://beekeep.pcs.usp.br/, in
Portuguese). Participants provided information about the stingless bee nest, location,
date, time, and weather conditions for each monitoring performed. The monitoring
consisted of recording videos of the entrance of nests and then counting the number
of bees in activity. The final class of the course consisted in discussing with the
citizen scientists the results obtained regarding both the biological findings and their
experience in participating in a scientific project.

2.2. Ethical aspects

Participants were required to complete a Free and Informed Consent Form
(CAEE 86 53398721.9.0000.5390) and only responses from participants who agreed
with the terms were included in the analyses.

2.3. Dimensions of learning assessment

The profile of the citizen scientists and some aspects related to seven
learning dimensions were assessed, considering the framework proposed by Phillips
et al. [24], adapted to our context, as follows:

116
Profile: The first part of the survey aimed to evaluate some characteristics of citizen
scientists to establish unique profiles and measure how these characteristics impact
their learning.
Knowledge of the Nature of Science: Nature of Science (epistemological
comprehension of scientific thinking). Our evaluation specifically focused on
determining whether citizen scientists had an understanding of certain scientific
project’s characteristics, mainly in Natural Sciences, such as the fact that they do not
always rely on hypothesis testing. Self-Efficacy: Self-efficacy is an essential
component in environmental citizenship, which depends on an individual’s beliefs
about their capabilities to learn specific content, knowledge, and sufficient skills to
bring positive change in their communities or themselves. Self-efficacy is sometimes
referred to as “perceived competence” or “perceived behavioral control” [25]. We
evaluated the participants’ perception of Self-Efficacy related to skills in bee biology,
bee monitoring, and meliponiculture.
Knowledge of Scientific Process: Understanding scientific data collection and
analysis processes using the stingless bees’ flying activity monitoring protocol as a
case study.
Knowledge of Scientific Content: Scientific literacy and understanding of the
theoretical content relative to the course. The assessment had a focus on content
related to bees and meliponiculture.
Interest: Interest is a complex construct encompassing cognitive, affective, and
behavioral domains. Interest is also a key precursor to deeper engagement in
participative decision-making processes in science [26]. We assessed the
participants’ interests regarding bees, sustainability, science, and social interactions.
Behavior and Stewardship: Behavior and Stewardship are an aggregate of
outcomes that results from participation in Citizen Science. This domain is intricately
linked to non-epistemic values and volunteers’ previous experience and is aimed to
evaluate changes in their behavior towards civic actions in society. Our evaluation
specifically focused on how citizen scientists prioritized various dimensions of
conservation, including environmental, social, and economic factors. We also sought
to determine whether they were committed to contributing to the conservation of
bees and whether they recognized the impact of bees on their overall quality of life.
Motivation: Motivation in citizen science is dynamic and complex, being both an
input and outcome, influenced by the participant's interest, recognition, and

117
attribution [27]. We assessed the reasons leading the participants to engage in the
course and perform the citizen science protocol as well.

2.4. Data collection instrument - Pre and Post Questionnaires

The pre and post-questionnaires (Table 1) were developed and refined over the
course of three editions of the Outreach Course in Meliponiculture and Citizen
Science, held in the second half of 2020, as well as the first and second half of 2021.
All questions were designed with closed-ended formats, including multiple-choice
and Likert scale responses. The questionnaires were further improved after
analyzing the results obtained from these three editions. Only the results from the
fourth edition of the course were considered in the present study. To evaluate the
internal consistency among groups of questions within the same learning dimension,
Cronbach’s α [28] was employed. The α coefficient was calculated by aggregating
responses before and after the course using the R package psych. The dimension
with the lowest Cronbach’s α was Knowledge of Scientific Content, with values of
0.35 (rather reliable) and 0.40 (rather reliable) before and after the course,
respectively. The dimensions exhibiting quite reliable Cronbach’s α values were
Knowledge of the Nature of Science (initial: 0.52), Knowledge of the Scientific
Process (initial: 0.44), and Behavior and Stewardship (initial: 0.56; final: 0.57). All
other Cronbach’s α were classified as reliable and very reliable (Table 2). The
dimensions with the highest values were Self-Efficacy (initial: 0.86; final: 0.83) and
Interest (initial: 0.81; final: 0.85). Despite attempts to regroup or remove questions,
Cronbach’s α did not exhibit improvement for dimensions that were already
considered rather reliable or quite reliable. Hence, we decided to retain them as the
final version of the questionnaires (Table 1) and explore potential modifications for
future course editions.

2.5. Statistical Analyses

Regarding the questions on motivation, the relative percentages of participants who

selected each alternative were calculated, with ni being the number of participants
who chose a given alternative in the previous questionnaire and nf the number of

118
participants who chose the same alternative in the subsequent questionnaire.
Relative percentages are calculated as the increase or decrease between ni and nf.
For each of the remaining learning dimensions (Section 2.3 and Table 1), the mean
score for the group of questions was computed, as we intended to assess changes
in learning across dimensions and not for individual questions. We first assessed
whether the answers for each question were positively correlated with the Likert
scale (i.e. if the highest score indicated the most positive or the desired answer for
that question) and, for those negatively correlated questions, we inverted the scores
before running the analysis. In order to test whether there were changes in mean
scores after the course (i.e. learning) and if those changes were affected by the
participant’s profile, we employed linear mixed effects models and tested our
hypotheses using model selection. The mean score was fitted as the response
variable, with time (before and after the course) and its interaction with six profile
variables as predictors (age, gender, level of education, area of knowledge, contact
with nature, and if the person was a beekeeper). The participant’s identity was
included as a random factor to account for dependencies in data. All models were
fitted using the R package lmerTest [29]. The dispersion was diagnosed by
comparing the residuals of the fitted model to simulated residuals (n=10,000
simulations), using the R package DHARMa [30]. Based on these simulations, we
also inspected the presence of outliers, which were removed from our dataset. Both
before and after outliers were removed from the dataset and final sample sizes for
each analysis are shown in Table 3. Model selection was performed by deleting each
interaction term with an F-test using Satterthwaite’s method to compare the full
model to the reduced one. This procedure was repeated until the best model was
reached.

119
Table 1. Aspects of the participants’ profile and indicators of each dimension of
learning evaluated in the pre and post-questionnaires.
Profile
-Gender
-Age
-Level of education
-Area of knowledge (higher education
only)
-Contact with nature (Frequency)
-Meliponist (yes/no)
Self-Efficacy 1 to 10
-If you were invited to participate in a
bee monitoring project, how capable of
helping that project do you think you
would be?
-How capable do you think you would
be if you were asked to take care of a
stingless bee colony (protect, feed)?
-If you were asked to perform stingless
bee colony management (such as
capable transfers and multiplications),
how capable of performing these tasks
do you think you would be?
-What do you think is your ability to
identify a stingless bee species?
-How capable do you think you are of
performing stingless bee counting on
Nature of Science Knowledge (level
of agreement: 1 to 5)
-For a project to be considered
scientific, it must:
-Be complex
-Be difficult
-Propose a way to analyze the data
-Try to answer a question that is of
society’s interest
-Have testable hypotheses
Scientific knowledge and process 1
to 5
-Stingless bees collect resources evenly
throughout the day
-On cold and rainy days the external
activity of bees decreases
-My presence close to the nest does not
interfere with the bees' flight activity
-The time of year does not influence the
flight activity of bees
-To monitor the bees' flight activity, I
must feed them before
-It is possible to monitor the flight
activity of bees, both on cold and hot
days
-To monitor the bee flight activity, I must
choose the time of highest activity
120
the flight in a video?
-How capable do you think you are of
asking a scientific question for a
research project?
Scientific content 1 to 5
-Cultivate flowers is important for
beekeeping
-I should only feed the bees in times of
scarcity of resources
-I should only keep bees that occur in
my region
-There is no problem transporting the
nests over long distances
-Meliponiculture has low potential to be
a sustainable activity
Behavior and Stewardship 1 to 5
-Bees contribute to my well-being
-Bees contribute to my quality of life
-I feel responsible for the conservation
of bees
-I would like to know how I can help
conserve bees
-Information about the death of bees is
exaggerated
-A lot of money is spent on bee
research
-Only professional scientists should
monitor the flight activity of stingless
bees
-Monitoring bee flight activity during a
swarm can generate unreliable data
-Laboratory equipment is required to
monitor the flight activity of stingless
bees
Interest 1 to 5
-I like to study bees
-I like to keep bees
-I like science
-I like to do scientific research
-I like to interact with new people
-I am interested in the protection of bee
species
-I am interested in the subject of
"sustainability"
Motivation (Up to three options)
-Learn more about bees
-Contribute to scientific research on
bees
-Meet people who deal with bees daily
-Meet researchers working with bees
-Helping in the conservation of bee
species
-Contribute to the development of public
policies
-Do something relevant to society
121
 -We must conserve bees because they
provide products that we use
-It is more important to guarantee the
income of poor people than to preserve
bees
-It is more important to build houses for
those in need than to preserve bees
-It is more important to produce food
than to preserve natural habitats
-Carry out a fun activity
-Learn to monitor bee nests
-Answering questions I have about bees
and their nests
-Increase the productivity of my
meliponary
- increase my income

Table 2. Cronbach’s α for each learning dimension in the pre (initial) and post (final)
questionnaires.

Learning Dimension Cronbach's Cronbach's

alpha (initial) alpha (final)

Nature of Science Knowledge 0.52 0.62

Self-Efficacy 0.86 0.83

Scientific Knowledge and Process 0.44 0.64

Scientific Content 0.35 0.40

Interest 0.81 0.85

Behavior and Stewardship 0.56 0.57

122
Table 3. Best models describing each learning dimension, including the number of
observations (N) and participants (P), interactions between fixed factors, degrees of
freedom (df), F-test value (F) and P-value. Time is the periods before and after the
training course. Beekeeper means if the participant was a meliponist or not.

Learning Number of Fixed factor Df F P-value

dimension observations
(participants
)

Nature of 1706 (853) Time : Level of education 3 2.75 0.04

science
Time : Area of knowledge 4 2.66 0.03
knowledge

Self-efficacy 1714 (857) Time : Gender 3 4.32 < 0.01

Time : Beekeeper 1 19.60 < 0.001

Science 1710 (855) Time : Area of knowledge 4 3.08 0.01

Knowlwdge
Time : Beekeeper 1 7.96 < 0.01
and Process

Scientific 1710 (855) Time : Beekeeper 1 39.07 < 0.001

Content

Interest 1672 (836) Time: Beekeeper 1 8.26 < 0.01

Behavior and 1688 (845) Time 1 1.91 <0.001

Stewardship

123
3. Results

The course had 1,112 participants who completed all classes and activities, 919
participants agreed to the informed consent form and authorized the use of their
anonymized responses for scientific research. We also excluded the underages,
reaching n = 835.

3.1. General Profile

The general profile of the participants shows some particularities of the search for
the course.The level of education of the majority of the participants was higher
education (43.71% ongoing or complete), mainly in Humanities (28.50%) or
Biological Sciences (27.43%), and most of them have a graduate degree (38.80%).
About two-thirds were male, and the frequency of volunteers’ contact with nature
was high. More than half of the participants were meliponists (52.34%), and the
majority of the participants were between 40 and 59 years old (Table 4).

Table 4. Participants General Profile (self-declared demographic aspects of the

course participants shown in percentages by the group).

Gender Age
Female 20 to 39 38.32%
34.01% 40 to 59 46.23%
Male 65.75% 60 to 79 15.45%
Non-binary 0.12%
I prefer not to tell 0.12%

Level of Education Area of Knowledge (Higher

Education)
Basic Education 17.49%
Higher Education (Ongoing) Exact Sciences 17.72%
13.29% Humanities 28.50%
Higher Education (Complete) Biological Sciences 27.43%

124
 30.42% Graduate Interdisciplinary Sciences 7.31%
38.80% No Higher Education
19.28%

Beekeeper Contact with Nature

Rarely 9.58%
Meliponist Occasionally 25.27%
52.34% Frequently 65.15%
Non-meliponist
47.66%

All groups exhibited an increase in their average general scores after the
course, suggesting positive learning.

3.2. Knowledge of the Nature of Science

The knowledge of the Nature of science was affected by time and its
interaction with the level of education and the area of knowledge (Table 5).
Participants with basic education showed higher learning than those with higher
education (Figure 1a). Background regarding the area of knowledge also affected
the scores (Figure 1b), with participants from exact sciences and humanities
showing lower average scores. Positive learning was observed for the participants of
all areas of knowledge, with those with interdisciplinary sciences backgrounds
exhibiting higher learning.

125
Table 5. Parameter estimates for the best model describing the mean scores in
Knowledge of the Nature of Science. Estimates for time are given with “post
questionnaires” as comparison level, estimates for level of education are given with
“basic education” as comparison level and estimates for area of knowledge are given
with “biological sciences” as comparison level. Std. Error is the standard error, df the
degrees of freedom and P-values were computed using the Satterthwaite
approximation.

Estimate Std. df Pr(>|t|)

Error (Interce
pt)

(Intercept) 4.01816 0.13251 1407.327 < 2e-16

57

TimeD 0.49966 0.13921 845.0000 0.00035

0 1

Level of Education Higher Education 0.19216 0.12232 1407.327 0.116433

(Ongoing) 57

Level of Education Higher Education 0.23282 0.13448 1407.327 0.083611

(Complete) 57

Level of Education Graduate 0.33918 0.13322 1407.327 0.011000

57

Area of Knowledge Exact Sciences -0.14419 0.05900 1407.327 0.01464

55 9

Area of Knowledge Humanities -0.19050 0.05352 1407.327 0.00038

55 4

Area Of Knowledge Interdisciplinary -0.14194 0.08096 1407.327 0.07978

Sciences 55 5

Area Of Knowledge No Higher -0.04127 0.12803 1407.327 0.74724

126
 Education 57 4

TimeD: Level of Education Higher -0.36105 0.12851 845.0000 0.00507

Education (Ongoing) 0 6

TimeD: Level of Education Higher -0.38432 0.14127 845.0000 0.00665

Education (Complete) 0 5

TimeD:Level of Education Graduate -0.36792 0.13995 845.0000 0.00872

0 2

TimeD:Area of Knowledge Exact 0.10865 0.06198 845.0000 0.07997

Sciences 0 8

TimeD:Area of Knowledge Humanities 0.10769 0.05623 845.0000 0.05581

0 3

TimeD:Area Of Knowledge 0.17176 0.08506 845.0000 0.04376

Interdisciplinary Sciences 0 0

TimeD:Area Of Knowledge No Higher -0.20055 0.13450 845.0000 0.13633

Education 0 0

3.3. Self-Efficacy

Self-Efficacy was affected by time and its interaction with Gender and if the
participant is a meliponist or not (Table 6). Higher initial scores were observed for
males and meliponists, when compared to females and non-meliponists,
respectively. All groups exhibited an increase in their average scores in the
post-questionnaires, revealing an increase in their perception of self-efficacy.
However, females (Figure 2a) and non-meliponist (Figure 2b) participants showed
higher levels of increase than males and meliponists, respectively.

127
3.4. Knowledge of the Scientific Process

Knowledge of the Scientific Process was affected by the interaction of time

and area of knowledge or if the participant is a meliponist or not (Table 7).
Participants from the exact sciences, humanities, and with no higher education
showed lower initial average scores when compared to participants from the
biological sciences. In addition, meliponists showed higher initial scores than
non-meliponists. All groups increased their scores after the course, suggesting
positive learning. However, meliponists (Figure 3a) and participants from the exact
sciences and interdisciplinary sciences areas of knowledge (Figure 3b) showed
higher learning than non-meliponists and those from the biological sciences,
respectively.

(a) Level of Education (b) Area of Knowledge (higher education)

Figure 1. Average scores for “Knowledge of the Nature of Science” dimension in the
pre (white bars) and post (gray bars) course questionnaires.

128
Table 6. Parameter estimates for the best model describing the mean scores for
"Self-Efficacy" dimension. Estimates for time are given with “post questionnaires” as
comparison level, estimates for level of gender are given with “female” as
comparison level and estimates for beekeeper are given with “non-meliponist” as
comparison level. Std. Error is the standard error, df the degrees of freedom and
P-values were computed using the Satterthwaite approximation. The values in bold
are significantly different from zero.

Estimate Std. Error df Pr(>|t|)

(Intercept)

(Intercept) 5.85224 0.09807 1429.12235 < 0.001

TimeD 2.15555 0.10392 852.00000 < 0.001

Gender Male 0.46731 0.11424 1429.12235 < 0.001

Gender Non-binary 1.31443 1.54826 1429.12235 0.40

Gender I Prefer not -1.32321 1.54933 1429.12235 0.39

to say

Beekeeper 0.80430 0.10869 1429.12235 < 0.001

Meliponist

TimeD:Gender Male -0.41445 0.12106 852.00000 < 0.001

TimeD:Gender -0.98889 1.64062 852.00000 0.546

Non-binary

TimeD:Gender I 1.35440 1.64176 852.00000 0.409

Prefer not to say

TimeD:Beekeeper -0.50995 0.11517 852.00000 < 0.001

Meliponist

129
 (a) for each Gender (b) meliponists and non-meliponists
Figure 2. Average scores for “Self-Efficacy” dimension in the pre (white bars) and
post (gray bars) course questionnaires.

Table 7. Parameter estimates for the best model describing the mean scores in
Knowledge of the Scientific Process. Estimates for time are given with “post
questionnaires” as comparison level, estimates for level of education are given with
“basic education” as comparison level and estimates for area of knowledge are given
with “biological sciences” as comparison level. Std.estimates for beekeeper are
given with “non-meliponist” as comparison level. Error is the standard error, df the
degrees of freedom and P-values were computed using the Satterthwaite
approximation. The values in bold are significantly different from zero.

Estimate Std. Error df Pr(>|t|)

(Intercept)

(Intercept) 3.63841 0.03361 1476.11446 < 0.001

TimeD 0.56258 0.03719 849.00000 < 0.001

Area of Knowledge Exact -0.10642 0.04806 1476.11446 < 0.01

130
 Sciences

Area of Knowledge -0.12775 0.04275 1476.11446 < 0.001

Humanities

Area Of Knowledge -0.08810 0.06604 1476.11446 0.182405

Interdisciplinary Sciences

Area Of Knowledge No -0.18721 0.04821 1476.11446 < 0.001

Higher Education

Beekeeper Meliponist 0.15108 0.03216 1476.11446 < 0.001

TimeD:Area of Knowledge 0.16424 0.05318 849.00000 < 0.001

Exact Sciences

TimeD:Area of Knowledge 0.01938 0.04730 849.00000 0.682199

Humanities

Area Of Knowledge 0.12808 0.07308 849.00000 0.080016

Interdisciplinary Sciences

Area Of Knowledge No 0.03894 0.05335 849.00000 0.465589

Higher Education

Beekeeper Meliponist -0.10042 0.03559 849.00000 < 0.001

3.5. Knowledge of Scientific Content

Knowledge of Scientific Content was affected by the interaction of time and

being a beekeeper (Table 8). Meliponists showed higher average scores in the
pre-questionnaires than non-meliponists. All groups increased their scores after the
course, suggesting positive learning. However, non-meliponist participants showed
higher learning than meliponists (Figure 4a).

131
Table 8. Parameter estimates for the best model describing the mean scores in
Knowledge of Scientific Content. Estimates for time are given with “post
questionnaires” as comparison level. Std.estimates for beekeeper are given with
“non-meliponist” as comparison level. Error is the standard error, df the degrees of
freedom and P-values were computed using the Satterthwaite approximation. The
values in bold are significantly different from zero.

Estimate Std. Error df Pr(>|t|)

(Intercept)

(Intercept) 3.99951 0.02531 1483.22777 < 0.001

TimeD 0.41460 0.02801 853.00000 < 0.001

Beekeeper 0.30229 0.03512 1483.22777 < 0.001

Meliponist

TimeD:Beekeeper -0.24298 0.03887 853.00000 < 0.001

Meliponist

3.6. Interest

Interest was affected by the interaction of time and being a beekeeper (Table 9).
Meliponists
showed higher initial scores than non-meliponists. All groups exhibited an increase in
their scores after the course, suggesting an increase in interest along the course.
However, non-meliponist participants showed a higher increase in interest than did
the meliponists (Figure 4b).

3.7. Behavior and Stewardship

Behavior and stewardship scores were affected by time (Table 10), with a decrease
in the average scores after the course. In contrast to the other learning dimensions,
no profile variable affected the answers related to behavior and stewardship.

132
 (a) Knowledge of Scientific Content (b) Interest
Figure 4. Average scores for “Knowledge of Scientific Content” (a) and “Interest” (b)
dimension in the pre (white bars) and post (gray bars) course questionnaires,
considering meliponists and non-meliponists.

Table 9. Parameter estimates for the best model describing the mean scores in
Interest. Estimates for time are given with “post questionnaires” as comparison level,
Std. estimates for beekeeper are given with “non-meliponist” as comparison level.
Error is the standard error, df the degrees of freedom and P-values were computed
using the Satterthwaite approximation. The values in bold are significantly different
from zero.

Estimate Std. Error df Pr(>|t|) (Intercept)

(Intercept) 4.39444 0.01961 1163.30395 < 0.001

TimeD 0.08621 0.01621 834.00002 < 0.001

Beekeeper 0.17666 0.02735 1163.30395 < 0.001

Meliponist

TimeD:Beekeeper -0.06494 0.02260 834.00002 < 0.001

Meliponist

133
Table 10. Parameter estimates for the best model describing the mean scores in
Behavior and Stewardship. Estimates for time are given with “post questionnaires”
as comparison level, Error is the standard error, df the degrees of freedom and
P-values were computed using the Satterthwaite approximation. The values in bold
are significantly different from zero.

Estimate Std. Error df Pr(>|t|)

(Intercept)

(Intercept) 3.14206 0.01209 1410.20670 < 0.001

TimeD -0.0509 0.01277 843.00001 < 0.001

3.8. Motivation

Regarding the Motivation dimension, only three alternatives had a higher number of
responses in the post questionnaire than in the initial one: i. Answer questions about
bees and their nests (46.51% growth); ii. Carry out a fun activity (growth of 22.73%);
and iii. Increase the productivity of my meliponary (growth of 12.50%). The other
alternatives showed a decrease, with a reduction of 100% for “Learn to monitor bee
nests”. The values obtained for the other alternatives were: Accomplish something
relevant to society (-30.73%); Meet researchers working on bees (-27.78%);
Contribute to the development of public policies (-26.35%); Meet people who deal
with bees daily (-21.05%); Learn more about bees (-20.74%); Increase my income
(-20%); Help in the conservation of bee species (-19.86%); Contribute to scientific
research on bees (-10.61%).

134
Table 11. Number of participants for each Motivation Variable

Selected Answer ni nf Relative %

Learn more about bees 651 516 -20.74%

Contribute to scientific research on bees 443 396 -10.61%

Meet people who deal with bees daily 76 60 -21.05%

Meet researchers working on bees 108 78 -27.78%

Helping in the conservation of bee species 700 561 -19.86%

Contribute to the development of public policies 148 109 -26.35%

Do something relevant to society 192 133 -30.73%

Carry out a fun activity 22 27 22.73%

Learn to monitor bee nests 85 0 -100%

Answering questions I have about bees and their 43 63 46.51%

nests

Increase the productivity of my meliponary 48 54 12.50%

Increase my income 30 24 -20%

4. Discussion

The present study aimed to explore the potential of training and active
participation in citizen science projects for promoting learning among participants.
Our results revealed that several profile variables had a limited impact on the
learning outcomes. Few profile variables of the participants were influential in the
increase of the analyzed learning dimensions. Only the Level of Education, the Area
of Knowledge, and whether the participant was a meliponist influenced the learning.

135
The participant’s gender was only relevant for the variation in the perception of
self-efficacy, but it did not influence any other learning dimension.
The finding that the level of education influenced learning outcomes is not
surprising, as previous research has consistently shown that education is positively
correlated with learning and knowledge acquisition [31]. However, the study also
found that the area of knowledge was a significant factor, suggesting that
participants with prior knowledge or interest in the specific topic of the citizen science
project may have an advantage in understanding the scientific concepts presented to
them. Interestingly, gender was found to be relevant only in the self-efficacy
dimension but not in any other learning dimension. This finding is consistent with
previous research showing that self-efficacy beliefs are influenced by gender, with
males sometimes reporting higher self-efficacy in science-related tasks than females
[32,33]. However, the lack of gender differences in the other learning dimensions
243 measured in this study suggests that citizen science projects may effectively
promote science education regardless of gender. On the other hand, gender equity
actions could be applied in this initiative to increase women’s participation, as most
participants were male.

4.1. Knowledge of the Nature of Science

Few citizen science projects have attempted to study the understanding of the
Nature of Science [24]. Our study shows that the profile variables related to
education were correlated to the observed differences in learning of the “Knowledge
of the Nature of Science” dimension. The scores obtained in the final questionnaires
were increased for participants of all levels of education and different areas of
knowledge. The higher scores for those in the Biological Sciences area of
Knowledge were not expected since the questions dealt with scientific fundamentals
for all areas of knowledge. The higher increase in the Knowledge of the Nature of
Science scores of participants with no higher education indicates that the training
course’s next editions should prioritize this audience, aiming to contribute to their
scientific literacy. Golumbic et al. [34] found similar results in their evaluation of
‘Sensing the Air’ citizen science project. Their study found that citizen scientists with
higher levels of education achieved higher average scores. On the other hand, more
experienced citizen scientists with lower levels of formal education also achieved

136
high scores. Both experience’s results suggest that while formal scientific education
is important, other experiences like participation in a citizen science project may
improve the scientific literacy of the participants.
Price and Lee [35], in a study that used the Nature of Scientific Knowledge
Scale (NSKS), observed that epistemological convictions relating to the nature of
science were notably enhanced. The participants debated specifically whether the
scientific process involves creativity and adjusted their beliefs regarding the aim of
science to create universal laws. Similarly, the study of Masters et al. [36] indicated
that the participants gained a fresh outlook on scientific research.

4.2. Self-Efficacy

The average self-efficacy scores for the male gender were higher than for the
female gender, and the female gender had a higher increase in the scores for the
Self-Efficacy dimension. The fact that gender was not relevant for all the other
evaluated learning dimensions indicates that Self-Efficacy or “perceived
competence” does not necessarily reflect the real knowledge or skills of the studied
group. The higher self-efficacy average scores for meliponists was expected
because of their previous experience in subjects related to bees, meliponiculture,
and contribution to scientific research. The higher increase observed in the scores of
non-meliponists also indicates that the training course achieved its expected
objective of making the participants with no or low previous experience feel more
confident.
Huang [37], in a meta-analysis of 187 studies containing 247 independent
studies, found that males have significantly higher levels of perception of
self-efficacy in mathematics, computer, and social sciences than females. Females
showed a higher perception of self-efficacy than males for language and arts.
Altunsoy et al. [38] found that male biology teacher candidates have significantly
higher levels of perception of self-efficacy than female candidates. Once males did
not have higher learning scores than females in any other dimension analyzed, the
differences in the perception of self-efficacy by gender in our results are probably a
consequence of gender inequalities in Brazil. Hiller [39] conducted a
classroom-based study comparing middle school students engaged in a horseshoe

137
crab citizen science project to a control group and found that the participating
students exhibited
greater improvements in their perception of self-efficacy. Conversely, a study by
Price and Lee [35] on an online astronomy project reported a significant decrease in
science perception of self-efficacy among participants, possibly due to increased
awareness of their lack of knowledge on the topic. According to Crall et al. [40],
self-efficacy is critical in carrying out project activities and empowering individuals to
undertake future environmental stewardship actions. Based on our findings,
strategies to assure female participants of improvement in their perception of
self-efficacy can be a way to promote gender equality in meliponiculture, an activity
carried out majoritarian by males [21]. In a broader context, evaluating citizen
scientists’ self-efficacy should consider profile aspects to base the proposition of
strategies that ensure diversity and equity in citizen science projects.

4.3. Knowledge of Scientific Process

The questions applied to the training course participants in this learning

dimension were mainly related to their understanding of the steps of scientific inquiry.
According to Freitag et al. [41] and Davis et al. [42] the training and evaluation of
scientific skills is a strategy to increase the credibility of citizen science-generated
data. However, many projects only evaluate the skills related to data collection [43].
The higher scores obtained by the participants from the Biological Sciences area of
knowledge were expected because we considered aspects in the questionnaires that
were strongly related to the Flight Activity Monitoring protocol and biological factors.
The lower initial scores for those participants from the Humanities area of knowledge
or no higher education indicates the great potential of the training course in making
the scientific process easy to understand for people from areas of knowledge not so
closely related to the Biological and Exact Sciences. A higher increase in the
non-meliponists’ scores was expected since the course presented the basic
concepts of the scientific inquiry process, e.g., research questions, hypothesis
testing, monitoring protocol, data visualization, and analysis. Those participants who
were meliponists already had some knowledge about the scientific inquiry process
since many of them had already collaborated with scientific research, as reported by
them in the pre-questionnaire. Some studies that evaluated the learning of the

138
Scientific Process on citizen science projects found no change in participants [11,44]
while others showed an increase [45,46]. The increase in Cronbach’s α in the
post-questionnaire indicates that the participants were more consistent with their
answers at the course’s end than at the beginning, which can be considered positive.

4.4. Knowledge of Scientific Content

In this particular learning dimension, it was expected that meliponists would

achieve higher initial scores, given that the questions primarily focused on bees and
meliponiculture. The greater learning outcomes observed among non-meliponists
emphasize the significance of training initiatives aimed at enhancing the
development of meliponiculture, which aligns with the findings of Barbiéri and
Francoy [19]. Various studies examining the knowledge of scientific content in citizen
science initiatives have yielded interesting results. For instance, Masters et al. [36]
investigated the relationship between participants’ engagement in citizen science
projects and their project-specific knowledge using quizzes. They discovered a
positive correlation between engagement and knowledge, with the strength of the
association varying depending on the project’s subject matter. Notably, astronomy
projects exhibited a stronger association compared to other projects, suggesting that
the extent of public involvement in a project can influence knowledge outcomes.
Aristeidou et al. [47] demonstrated that some participants made advancements in
their topic-specific lexicon by incorporating scientific terminology and using
vocabulary related to the specific scientific investigation. Furthermore, the study
showed that some participants corrected misconceptions they held regarding
everyday aspects of the scientific topic, such as the belief that there is no extreme
weather in southern regions. The analysis of identification data in Scanlon et al. [48]
suggests that the level of engagement positively correlates with the number of
accurate identifications made by the participants, indicating improvement over time.
However, the self-reported learning impact of participants in the study conducted by
Vohland et al. [1] was modest, with participants indicating that they learned
“somewhat” about the scientific topic concerning aerosols’ health and environmental
impact. In the present study, despite the increase in the value of Cronbach’s α in the
post-questionnaire (from 0.35 to 0.40), there is still a need to enhance the strategies

139
employed in the course for effectively teaching beekeeping and meliponiculture as a
sustainable activity.

4.5. Interest

The interest scores grew among the participants of our study. The only profile
variable affecting the interest dimension was whether the participant was a
meliponist. Those who were meliponists practically did not change their interest,
which was already high, after participating in the course. The higher increase of
interest for non-meliponists indicates they are a good target group to recruit for
citizen science projects. Price and Lee [35] documented a heightened interest in
science among individuals who participated in Citizen Sky, particularly those who
engaged in online social activities. On the other hand, Crall et al. [40] investigated
the factors that lead to participation in citizen science projects and found that a
general interest in science was not a significant motivator for involvement. Rather, an
interest in particular nature-related subjects, such as butterflies, was identified as a
driving force for engagement and an incentive for adopting more complex data
collection protocols in the French Garden Butterflies Watch project [49]. Furthermore,
previous research has revealed that the interest in utilizing natural resources can
strongly influence an individual’s participation in decision-making related to natural
resource management [50]. Phillips et al. [24] highlights that evaluating general
interest in science may be less effective than measuring specific science topics.
However, it is also plausible that participation in citizen science may not substantially
alter an individual’s pre-existing interests in specific topics.
According to Vohland et al. [1], the interest dimension is more frequently
assessed as a previous interest leading to participation. People can sometimes
become more interested in the subject after engaging in the citizen science project.
Few studies assess the interest after participating in the project. De Moor et al. [51]
describe an example of volunteers choosing to engage in more difficult tasks in a
transcription of a handwriting project after the first contact with easier objectives.
This shows the potential to recruit pre-selected groups of participants with previous
experience from other protocols to participate in new citizen science activities. Future
research may provide data to investigate the connection between citizen scientists’
engagement and their interests.

140
4.6. Behavior and Stewardship

Only the time variable has shown an effect on the observed changes in this
dimension. In other words, the selected profile variables were not relevant. This
happened probably because this dimension is intricately linked to non-epistemic and
personal values of the participants, like political, pro-equality, and pro-environmental
values. It is important to highlight that we were not considering any of the answers in
this dimension specifically as “right” or “wrong” and our results indicate that the
course had the effect of causing reflections about the evaluated topics. In
Ganzevoort and van den Born [52], no significant change was found in the Behavior
and Stewardship dimension variables after participating in the Dutch National Bee
Survey. Chase and Levine [53] points out that citizen science participation is more
valuable in previous positive behavior towards the environment reinforcement than in
producing change. According to Bela et al. [54], evaluation should be integrated as
an inherent step in the process to achieve greater reliability and transparency in
measuring the transformative effects of environmental citizen science projects.
Learning frequently occurs unintentionally and results in transformations at various
levels, including shifts in values, beliefs, emotions, and actions, so citizens and
scientists should be actively involved in the evaluation process through
self-assessment and reflection.

4.7. Motivation

Although participants in the same citizen science project share certain

interests and motivations, it is important not to treat them as homogeneous. While a
citizen scientist may engage in a project for various reasons, these motivations are
dynamic and can change over time [55]. Since the second questionnaire was
answered after completing the course and implementing the protocol, it was
expected that there could be changes in the participants’ motivations. The increase
in the scores of some options of this dimension in the second questionnaire indicates
that the project’s objectives were achieved, at least partially. Many citizen scientists
intend to continue monitoring the flight activity of stingless bee nests to answer
questions they have about the bees, indicating these citizen scientists approached
the science-making process. The increase in the score of the option “Carry out a fun

141
activity” is largely in line with the literature [24], since many citizen scientists engage
in projects as a form of leisure. The increase in the option “Increase the productivity
of my meliponary” may be due to the content of the course, in which flight activity
was presented as a sign of the strength of a colony. For options that have shown a
decrease, we can make some speculations. The option “Learn to monitor bee nests”
having reached a decrease of 100% may be a sign that the participants feel safe and
have learned to perform the protocol since the participant’s average Self-Efficacy
394 has increased. Other alternatives may have suffered a decrease due to similar
reasons since the participants had contact with researchers who work on bees and
with other people who work with bees on a daily basis, access to knowledge about
bees and meliponiculture, and contributed to scientific research on bees. Regarding
the alternatives “Contribute to the development of public policies”, “Increase my
income” and “Help in the conservation of bee species”, it is possible that the
participants have changed their perception about the importance of these
alternatives concerning the others.
Citizen science projects have the potential to facilitate learning and
engagement with a wide range of participants, including individuals from diverse
backgrounds and levels of education. However, it is important for project planners to
understand which participant profile variables may be influential in the learning
outcomes of the project. The results of this study suggest that the level of education,
area of knowledge, and participation in a specific field (in this case, meliponist) may
have a significant impact on the learning outcomes of citizen science participants.
Therefore, these variables should be considered when designing and implementing
citizen science projects, to ensure that the project is accessible and meaningful to a
diverse range of participants.
Furthermore, the finding that gender was only relevant in self-efficacy, but not
reflected in other learning dimensions, highlights the importance of carefully
selecting and measuring learning outcomes in citizen science projects. Project
planners should consider a range of learning outcomes and consider potential social
differences in learning and engagement when designing and implementing citizen
science projects.
Overall, understanding the influence of participant’s profile variables on the
learning outcomes of citizen science projects is crucial for effective project planning
and ensuring the accessibility and inclusivity of citizen science initiatives.

142
 5. Conclusion

Our results on assessing the influence of citizen scientists’ profiles on learning

showed that some characteristics of the course participants influenced their learning.
All groups achieved learning benefits from their participation in the course even
though some aspects of their profiles, such as their area of knowledge, level of
education, and previous experience in meliponiculture had influenced their
learning.The Meliponiculture and Citizen Science outreach course successfully
generated learning for the seven proposed learning dimensions. The assessment
provided insights into the #cidadãoasf citizen science project’s next steps and
upcoming training courses. We consider that the profile assessment of citizen
scientists is an interesting way to adequate and improve a project’s methods and
goals.
Citizen Science projects which provide training in a subject that is of
volunteers’ interest and matches their motivations may achieve greater success in
recruiting and retaining volunteers in the long term. Training in Citizen Science has
the potential to promote many different outcomes, such as answering research
questions with reliable data quality, generating environmental awareness, and
increasing scientific literacy. Citizen Science initiatives focused on meliponiculture
can also increase the participant’s general knowledge and practical skills regarding
bees and stingless beekeeping. The results of our study may be extrapolated to
other citizen science projects, and we strongly suggest evaluating the influence of
the volunteer’s profile on their learning outcomes as a subsidy to guide the
improvement of the quality of the projects.

Author Contributions: Conceptualization, N.G., S.K. and C.B.; methodology,

S.K., C.B., J.L., B.A., and N.G.; validation, S.K., C.B., N.G., J.L., B.A., and T.F.;
formal analysis, S.K. and J.L.; investigation, C.B., S.K., N.G. and J.L.; data curation,
J.L., S.K. and C.B.; writing–original draft preparation, C.B., N.G., J.L., S.K. and B.A.;
writing–review and editing, C.B., N.G., S.K., B.A., T.F. and A.S.; visualization, B.A.,
C.B., S.K., J.L. and N.G.; supervision, N.G., B.A., T.F., and A.S.; project
administration, A.S., T.F., N.G., and B.A.; funding acquisition, A.S.

143
 Funding: This work was partially funded by the PhD grant processes
88887.606651/2021-00 (C.B.) and 88882.333367/2019-01 (J.L.), the FAPESP
funding for the project "Salvaguardando serviços de polinização em um mundo em
mudança: teoria na prática (SURPASS2)", process n° 2018/14994-1 (A.S., B.A., T.F.
and N.G.) and 2019/26760-8 (SK), and by the Conselho Nacional de
Desenvolvimento Científico e Tecnológico - Brazil (CNPq),445 A.S. grant number
312605/2018-8.

Acknowledgments: We would like to express our sincere appreciation to the

funding agencies that supported our research. Their financial support was
instrumental in making this research possible. We would also like to acknowledge
the University of Sao Paulo’s Office of the Provost for Culture and Extension and the
School of Arts, Sciences, and Humanities of the University of Sao Paulo Commission
for Culture and Extension (CCEx) for their valuable contributions to this project. Their
assistance in coordinating and organizing our research activities was crucial to the
success of this project. Furthermore, we express our sincere thanks to
Meliponicultura.org for their support in conducting the course and to all the
participants who took part in it. Their enthusiastic participation and feedback helped
us to refine and improve our research. Conflicts of Interest: The authors declare no
conflict of interest.

References

1. Vohland, K.; Land-Zandstra, A.; Ceccaroni, L.; Lemmens, R.; Perelló, J.; Ponti,M.;
Samson, R.;Wagenknecht, K. The science of citizen science; Springer Nature, 2021.
2. Kobori, H.; Dickinson, J.L.; Washitani, I.; Sakurai, R.; Amano, T.; Komatsu, N.;
Kitamura, W.; Takagawa, S.;Koyama, K.; Ogawara, T.; others. Citizen science: a new
approach to advance ecology, education, and conservation. Ecological research
2016, 31, 1–19.
3. Evans, C.; Abrams, E.; Reitsma, R.; Roux, K.; Salmonsen, L.; Marra, P.P. The
neighborhood nestwatch program: participant outcomes of a citizen-science
ecological research project. Conservation biology 2005,19, 589–594.
4. Bonney, R.; Phillips, T.B.; Ballard, H.L.; Enck, J.W. Can citizen science enhance
public understanding of science? Public understanding of science 2016, 25, 2–16.

144
5. Kawabe, L.; Ghilardi-Lopes, N.; Turra, A.; Wyles, K. Citizen science in marine litter
research: A review. Marine Pollution Bulletin 2022, 182, 114011.
6. Wehn, U.; Gharesifard, M.; Ceccaroni, L.; Joyce, H.; Ajates, R.;Woods, S.; Bilbao,
A.; Parkinson, S.; Gold, M.; Wheatland, J. Impact assessment of citizen science:
state of the art and guiding principles for a consolidated approach. Sustainability
Science 2021, 16, 1683–1699.
7. Tweddle, J.C.; Robinson, L.D.; Pocock, M.; Roy, H.E. Guide to citizen science:
developing, implementing and evaluating citizen science to study biodiversity and the
environment in the UK; NERC/Centre for Ecology & Hydrology, 2012.
8. Turaga, R.M.R.; Howarth, R.B.; Borsuk, M.E. Pro-environmental behavior:
Rational choice meets moral motivation. Annals of the New York Academy of
Sciences 2010, 1185, 211–224.
9. Chao, S.H.; Jiang, J.Z.; Wei, K.C.; Ng, E.; Hsu, C.H.; Chiang, Y.T.; Fang, W.T.
Understanding pro-environmental behavior of citizen science: An exploratory study
of the bird survey in Taoyuan’s farm ponds project. Sustainability 2021, 13, 5126.
10. Wichmann, C.S.; Fischer, D.; Geiger, S.M.; Honorato-Zimmer, D.; Knickmeier, K.;
Kruse, K.; Sundermann, A.; Thiel, M. Promoting pro-environmental behavior through
citizen science? A case study with Chilean schoolchildren on marine plastic pollution.
Marine Policy 2022, 141, 105035.
11. Jordan, R.C.; Ballard, H.L.; Phillips, T.B. Key issues and new approaches for
evaluating citizen-science learning outcomes. Frontiers in Ecology and the
Environment 2012, 10, 307–309.
12. Kelemen-Finan, J.; Scheuch, M.; Winter, S. Contributions from citizen science to
science education: an examination of a biodiversity citizen science project with
schools in Central Europe. International Journal of Science Education 2018, 40,
2078–2098.
13. Aristeidou, M.; Herodotou, C. Online citizen science: A systematic review of
effects on learning and scientific literacy. Citizen Science: Theory and Practice 2020,
5, 1–12.
14. Oliveira, G.P.d. Estratégias multidimensionais para a avaliação da aprendizagem
em cursos on-line. Ensaio: avaliação e políticas públicas em Educação 2010, 18,
105–138.

145
15. Koffler, S.; Barbiéri, C.; Ghilardi-Lopes, N.P.; Leocadio, J.N.; Albertini, B.;
Francoy, T.M.; Saraiva, A.M. A buzz for sustainability and conservation: the growing
potential of citizen science studies on bees. Sustainability 2021, 13, 959.
16. MacPhail, V.J.; Gibson, S.D.; Colla, S.R. Community science participants gain
environmental awareness and contribute high quality data but improvements are
needed: insights from Bumble BeeWatch. PeerJ 496 2020, 8, e9141.
17. Leocadio, J.N.; Ghilardi-Lopes, N.P.; Koffler, S.; Barbiéri, C.; Francoy, T.M.;
Albertini, B.; Saraiva, A.M. Data Reliability in a Citizen Science Protocol for
Monitoring Stingless Bees Flight Activity. Insects 2021, 12, 766.
18. NogueiraNeta, P. Notas bionômicas sôbre osMeliponineos. Sôbre a
enxameagem(Hymenoptera, Apoidae). III. Arq. Mus. Nac.(Rio) 1954, 42, 419–452.
19. Barbiéri, C.; Francoy, T.M. Theoretical model for interdisciplinary analysis of
human activities:Meliponiculture as an activity that promotes sustainability. Ambiente
& Sociedade 2020, 23.
20. Barbieri Junior, C. Caracterização da meliponicultura e do perfil do meliponicultor
no estado de São Paulo:ameaças e estratégias de conservação de abelhas sem
ferrão. PhD thesis, Universidade de São Paulo, 2018.
21. Jaffe, R.; Pope, N.; Carvalho, A.T.; Maia, U.M.; Blochtein, B.; de Carvalho,
C.A.L.; Carvalho-Zilse, G.A.; Freitas, B.M.; Menezes, C.; de Fátima Ribeiro, M.;
others. Bees for development: Brazilian survey reveals how to optimize stingless
beekeeping. PloS one 2015, 10, e0121157.
22. Ardoin, N.M.; Bowers, A.W.; Gaillard, E. Environmental education outcomes for
conservation: A systematic review. Biological Conservation 2020, 241, 108224.
doi:https://doi.org/10.1016/j.biocon.2019.108224.
23. Børresen, S.T.; Ulimboka, R.; Nyahongo, J.; Ranke, P.S.; Skjaervø, G.R.;
Røskaft, E. The role of education in biodiversity conservation: Can knowledge and
understanding alter locals’ views and attitudes towards ecosystem services?
Environmental Education Research 2023, 29, 148–163.
24. Phillips, T.; Porticella, N.; Constas, M.; Bonney, R. A framework for articulating
and measuring individual learning outcomes from participation in citizen science.
Citizen Science: Theory and Practice 2018, 3.
25. Berkowitz, A.R.; Ford, M.E.; Brewer, C.A. A framework for integrating ecological
literacy, civics literacy, and environmental citizenship in environmental education.

146
Environmental education and advocacy: Changing perspectives of ecology and
education 2005, 227, 66.
26. Mejlgaard, N.; Stares, S. Participation and competence as joint components in a
cross-national analysis of scientific citizenship. Public Understanding of Science
2010, 19, 545–561.
27. Rotman, D.; Preece, J.; Hammock, J.; Procita, K.; Hansen, D.; Parr, C.; Lewis,
D.; Jacobs, D. Dynamic changes in motivation in collaborative citizen-science
projects. Proceedings of the ACM 2012 conference on computer supported
cooperative work, 2012, pp. 217–226.
28. Cronbach, L.J. Coefficient alpha and the internal structure of tests.
psychometrika 1951, 16, 297–334.
29. Kuznetsova, A.; Brockhoff, P.B.; Christensen, R.H. lmerTest package: tests in
linear mixed effects models.Journal of statistical software 2017, 82, 1–26. 526
30. Hartig, F.; Hartig, M.F. Package ‘DHARMa’. R package 2017.
31. Yli-Renko, H.; Autio, E.; Sapienza, H.J. Social capital, knowledge acquisition,
and knowledge exploitation in young technology-based firms. Strategic management
journal 2001, 22, 587–613.
32. Allaire-Duquette, G.; Chastenay, P.; Bouffard, T.; Bélanger, S.A.; Hernandez, O.;
Mahhou, M.A.; Giroux,530 P.; McMullin, S.; Desjarlais, E. Gender differences in
self-efficacy for programming narrowed after a 2-h science museum workshop.
Canadian Journal of Science, Mathematics and Technology Education 2022,532 22,
87–100.
33. Britner, S.L.; Pajares, F. Self-efficacy beliefs, motivation, race, and gender in
middle school science. Journal of women and Minorities in Science and Engineering
2001, 7.
34. Golumbic, Y.N.; Fishbain, B.; Baram-Tsabari, A. Science literacy in action:
understanding scientific data presented in a citizen science platform by non-expert
adults. International Journal of Science Education, Part B 2020, 10, 232–247.
35. Price, C.A.; Lee, H.S. Changes in participants’ scientific attitudes and
epistemological beliefs during an astronomical citizen science project. Journal of
Research in Science Teaching 2013, 50, 773–801.
36. Masters, K.; Oh, E.Y.; Cox, J.; Simmons, B.; Lintott, C.; Graham, G.; Greenhill,
A.; Holmes, K. Science learning via participation in online citizen science. arXiv
preprint arXiv:1601.05973 2016.

147
37. Huang, C. Gender differences in academic self-efficacy: A meta-analysis.
European journal of psychology of education 2013, 28, 1–35.
38. Altunsoy, S.; Çimen, O.; Ekici, G.; Atik, A.D.; Gökmen, A. An assessment of the
factors that influence biology teacher candidates’ levels of academic self-efficacy.
Procedia-Social and Behavioral Sciences 2010, 2, 2377–2382.
39. Hiller, 547 S. The effect of a horseshoe crab citizen science project on student
self-efficacy and career trajectories.Round table session at the annual research
conference of the American Educational Research Association, Vancouver, 2012.
40. Crall, A.W.; Jordan, R.; Holfelder, K.; Newman, G.J.; Graham, J.;Waller, D.M.
The impacts of an invasive species citizen science training program on participant
attitudes, behavior, and science literacy. Public understanding of Science 2013, 22,
745–764.
41. Freitag, A.; Meyer, R.; Whiteman, L. Strategies employed by citizen science
programs to increase the credibility of their data. Citizen Science: Theory and
Practice 2016, 1.
42. Davis, C.; Del Bianco, V.; Peterman, K.; Grover, A.; Phillips, T.; Becker-Klein, R.
Diverse and important ways evaluation can support and advance citizen science.
Citizen Science: Theory and Practice 2022, 7.
43. Stylinski, C.D.; Peterman, K.; Phillips, T.; Linhart, J.; Becker-Klein, R. Assessing
science inquiry skills of citizen science volunteers: a snapshot of the field.
International Journal of Science Education, Part B 2020,10, 77–92.
doi:10.1080/21548455.2020.1719288.
44. Brossard, D.; Lewenstein, B.; Bonney, R. Scientific knowledge and attitude
change: The impact of a citizen science project. International Journal of Science
Education 2005, 27, 1099–1121.
45. Ballard, H.; Phillips, T.; Robinson, L. Conservation outcomes of citizen science.
In Citizen Science; UCL Press, 2018.
46. Cronin, D.P.; Messemer, J.E. Elevating adult civic science literacy through a
renewed citizen science paradigm. Adult Learning 2013, 24, 143–150.
47. Aristeidou, M.; Scanlon, E.; Sharples, M. Profiles of engagement in online
communities of citizen science participation. Computers in Human Behavior 2017,
74, 246–256.

148
48. Scanlon, E.;Woods,W.; Clow, D. Informal participation in science in the UK:
identification, location and mobility with iSpot. Journal of Educational Technology &
Society 2014, 17, 58–71.
49. Cosquer, A.; Raymond, R.; Prevot-Julliard, A.C. Observations of everyday
biodiversity: a new perspective for conservation? Ecology and Society 2012, 17.
50. Danielsen, F.; Burgess, N.D.; Balmford, A.; Donald, P.F.; Funder, M.; Jones, J.P.;
Alviola, P.; Balete, D.S.;Blomley, T.; Brashares, J.; others. Local participation in
natural resource monitoring: a characterization of approaches. Conservation biology
2009, 23, 31–42.
51. De Moor, T.; Rijpma, A.; Prats López, M.; others. Dynamics of engagement in
citizen science: results from the “yes, i do!” project. Citizen Science: Theory and
Practice 2019, 4, 1–17.
52. Ganzevoort, W.; van den Born, R.J. Counting bees: Learning outcomes from
participation in the Dutch national bee survey. Sustainability 2021, 13, 4703.
53. Chase, S.K.; Levine, A. Citizen science: Exploring the potential of natural
resource monitoring programs to influence environmental attitudes and behaviors.
Conservation Letters 2018, 11, e12382.
54. Bela, G.; Peltola, T.; Young, J.C.; Balázs, B.; Arpin, I.; Pataki, G.; Hauck, J.;
Kelemen, E.; Kopperoinen, L.; Van Herzele, A.; others. Learning and the
transformative potential of citizen science. Conservation Biology 2016, 30, 990–999.
55. Geoghegan, H.; Dyke, A.; Pateman, R.; West, S.; Everett, G. Understanding
motivations for citizen science.
Final report on behalf of UKEOF, University of Reading, Stockholm Environment
Institute (University of York) and University of the West of England 2016.

© 2023 by the authors. Submitted to Sustainability for possible open access

publication under the terms and conditions of the Creative Commons Attribution (CC
BY) license (http://creativecommons.org/licenses/by/4.0/).

149
 8 - CONCLUSÃO

Tendo em vista o crescimento da popularidade das abordagens de ciência

cidadã em todo o mundo, o Sul Global, incluindo o Brasil, não pode ficar para trás
na construção deste novo modo de fazer ciência, que vem se mostrando tão
promissor. O Brasil é um país rico em diversidade em todos os aspectos, entre eles
o biológico, o ambiental, o étnico, o cultural e o social. A popularização da ciência
cidadã no Brasil não deve ser encarada apenas como mais uma forma de se gerar
conhecimento acadêmico, mas também como um caminho para gerar ganhos
mensuráveis nos domínios social, ambiental e cultural. A diversidade do Brasil trás
oportunidades para que o país, que hoje apresenta uma tímida, embora crescente
presença entre os países em que se desenvolvem projetos em ciência cidadã, se
torne uma potência e tome a vanguarda dessa forma de fazer ciência.
No espírito que a ciência cidadã traz, de ampliar e intensificar o
conhecimento entre a academia e os demais setores da sociedade, cabe aos
condutores dos projetos de ciência cidadã construir pontes entre os setores. Como
já foi identificado, os projetos de ciência cidadã tem grande potencial para se atingir
os Objetivos para o Desenvolvimento Sustentável da ONU, de forma que a
contribuição para estes objetivos podem estar atrelados aos objetivos do projeto
desde a sua concepção, gerando assim, mais um ponto de contato entre a pesquisa
científica e a sociedade.
O estreitamento dos laços entre a academia e os demais setores da
sociedade é essencial não só para a geração de conhecimento ou recrutamento de
cientistas cidadãos. Este estreitamento é importante para a geração de políticas
públicas em dois sentidos, o primeiro de consolidar a ciência cidadã no Brasil, e de
destacar a ciência cidadã como uma forma de contribuir para a construção de
políticas públicas participativas em diversos setores nos múltiplos domínios da
sustentabilidade.
De forma a possibilitar o aperfeiçoamento dos projetos em ciência cidadã,
mensurar seus resultados e direcionar seus esforços, a avaliação dos projetos é de
grande importância, seja no âmbito da qualidade dos dados gerados, seja para
avaliar as contrapartidas para as contribuições dos cientistas cidadãos ou mesmo
para a adequação dos projetos para melhor atender ou reavaliar seus objetivos.
Outro aspecto importante da avaliação dos projetos quando estes visam contribuir

150
com políticas públicas, é o de gerar indicadores e tornar as informações mais
acessíveis para os tomadores de decisão.
Em relação ao aprendizado dos cientistas cidadãos, a capacitação dos
mesmos é de fundamental importância, não só visando a produção de ciência de
qualidade, mas também para ampliar a sua autonomia em fazer ciência e encarar o
mundo com um olhar mais crítico. A avaliação do aprendizado na capacitação de
cientistas cidadãos é fundamental para identificar quais objetivos estão sendo
atingidos, quais podem ser melhorados e elaborar estratégias para o
aperfeiçoamento dos projetos.
A experiência com o Protocolo de Monitoramento da Atividade de Voo de
Abelhas sem ferrão, o primeiro projeto a ser implementado na plataforma BeeKeep,
trás uma série de lições que não se restringem apenas ao trabalho de ciência
cidadã com abelhas, mas que certamente devem ser consideradas na sua
implementação e execução. Entre as lições, temos a importância do retorno que é
dado pelos cientistas cidadãos para com os cientistas profissionais; seus relatos da
dificuldade em realizar a contagem da quantidade de abelhas que retornam
carregando pólen para dentro do ninho foram corroborados pela avaliação da
qualidade dos dados gerados pelos cientistas cidadãos quando comparados aos
cientistas profissionais. Tais relatos, somados à análise dos dados, possibilitaram a
implementação de uma função bastante simples que permite ao cientista cidadão
avisar quando não foi capaz de identificar a entrada de pólen. Tal exemplo ilustra
bem a importância do contato constante e bilateral entre os cientistas profissionais e
os cientistas cidadãos, sugestões interessantes podem partir dos voluntários mesmo
em projetos contributórios e participativos, aproximando-os da co-criação.
Outro aspecto importante que emergiu da experiência com o projeto foi o da
alta procura pelo Curso de Extensão em Meliponicultura e Ciência Cidadã da
Universidade de São Paulo. O conjunto da experiência demonstrou a demanda por
conhecimento relativo à temática das abelhas sem ferrão e capacitação em
meliponicultura, associado a um grande número de pessoas participando de um
projeto de ciência cidadã com abelhas. O curso teve procura de diversos níveis de
escolaridade e de contato com a meliponicultura, o que por si só, demonstra o
enorme potencial da criação de novos projetos de ciência cidadã no âmbito da
meliponicultura, envolvendo meliponicultores e não meliponicultores.

151
 Participantes de todos os perfis, independentemente de escolaridade, area
do conhecimento, idade, sendo ou não meliponicultores apresentaram tanto
aprendizado quanto aumento na auto-eficácia, também chamada de “competência
percebida”. A avaliação dos participantes trouxe luz para o caminho a ser trilhado
nos próximos trabalhos, bem como demonstrou por meio do maior aprendizado em
diversas variáveis e aumento do interesse dos meliponicultores após o projeto, que
os criadores de abelhas sem ferrão são um excelente grupo alvo para a realização
de projetos de ciência cidadã direcionados a abelhas sem ferrão.
O desenvolvimento de projetos de ciência cidadã ligados a meliponicultura e
abelhas sem ferrão se apresentam como uma enorme oportunidade de,
simultaneamente, ampliar o conhecimento sobre a biologia básica deste grupo de
insetos junto ao público geral e gerar conhecimento prático relativo ao
aperfeiçoamento de técnicas de manejo das abelhas. Sendo a meliponicultura,
quando bem orientada, uma atividade promotora da sustentabilidade, tais projetos
devem ser elaborados, executados e incentivados de forma a contribuir para a
construção de uma meliponicultura profundamente vinculada com a ciência e com
alta capacidade de contribuir para a implementação de políticas públicas voltadas a
alcançar os objetivos do desenvolvimento sustentável, bem como levar ao
desenvolvimento de projetos práticos, voltados para o desenvolvimento da criação
de abelhas sem ferrão no país e com participação expressiva dos meliponicultores,
grupo extremamente engajado e interessado em ajudar a construir um futuro melhor
para a atividade em nosso país.

152
 9 - REFERÊNCIAS

BONNEY, Rick et al. Can citizen science enhance public understanding of science?.
Public understanding of science, v. 25, n. 1, p. 2-16, 2016.

DICKINSON, Janis L.; ZUCKERBERG, Benjamin; BONTER, David N. Citizen

science as an ecological research tool: challenges and benefits. Annual review of
ecology, evolution, and systematics, v. 41, p. 149-172, 2010.

DUNN, Robert R.; BEASLEY, DeAnna E. Democratizing evolutionary biology,

lessons from insects. Current opinion in insect science, v. 18, p. 89-92, 2016.

GRÜTER, Christoph; GRÜTER, Christoph. Stingless bees: An overview. Stingless

Bees: Their Behaviour, Ecology and Evolution, p. 1-42, 2020.

LEPCZYK, Christopher A. et al. Symposium 18: Citizen science in ecology: the

intersection of research and education. The Bulletin of the Ecological Society of
America, v. 90, n. 3, p. 308-317, 2009.

LOPES, Marcio; FERREIRA, João Batista; DOS SANTOS, Gilberto. Abelhas

sem-ferrão. APA Agriculturas - v. 2 - no 4 , 2005.

MACPHAIL, Victoria J.; COLLA, Sheila R. Power of the people: A review of citizen
science programs for conservation. Biological Conservation, v. 249, p. 108739,
2020.

NOGUEIRA NETO, Paulo. Vida e criação de abelhas indígenas sem ferrão.

Nogueirapis, 1997.

POCOCK, M.J.O., CHAPMAN, D.S., SHEPPARD, L.J. & Roy, H.E. Choosing and
Using Citizen Science: a guide to when and how to use citizen science to
monitor biodiversity and the environment. Centre for Ecology & Hydrology.
Wallingford (2014).

153
SILVERTOWN, Jonathan. A new dawn for citizen science. Trends in ecology &
evolution, v. 24, n. 9, p. 467-471, 2009.

SUZUKI-OHNO, Yukari et al. Utilization of photographs taken by citizens for

estimating bumblebee distributions. Scientific reports, v. 7, n. 1, p. 1-11, 2017.

VENTURIERI, Giorgio Cristino et al. Caracterização, colheita, conservação e

embalagem de méis de abelhas indígenas sem ferrão. Belém: Embrapa
Amazônia Oriental, 2007.

WILCOVE, David S. Endangered species management: the US experience.

Conservation Biology for All, p. 220-235, 2010. In: SODHI, Navjot S.; EHRLICH,
Paul R. (Ed.). Conservation biology for all. Oxford University Press, 2010.

WOLOWSKI, Marina et al. Relatório temático sobre polinização, polinizadores e

produção de alimentos no Brasil. São Carlos, SP: Editora Cubo, 2019.

154