Algorithmic Problems of Group Theory, Their Complexity, and Cryptography

633
Algorithmic Problems of Group

Theory, Their Complexity, and
Applications to Cryptography
AMS Special Sessions
Algorithmic Problems of Group Theory and Their Complexity
January 9–10, 2013
San Diego, California
Algorithmic Problems of Group Theory and Applications

to Information Security
April 6–7, 2013
Boston College, Chestnut Hill, Massachusetts
Delaram Kahrobaei
Vladimir Shpilrain
Editors
American Mathematical Society

January 9–10, 2013

April 6–7, 2013
Delaram Kahrobaei
Vladimir Shpilrain
Editors
633

January 9–10, 2013

April 6–7, 2013
Delaram Kahrobaei
Vladimir Shpilrain
Editors
American Mathematical Society

Providence, Rhode Island
EDITORIAL COMMITTEE
Dennis DeTurck, Managing Editor
Michael Loss Kailash Misra Martin J. Strauss
2010 Mathematics Subject Classification. Primary 20-XX, 68-XX.
Library of Congress Cataloging-in-Publication Data

Algorithmic problems of group theory, their complexity, and applications to cryptography / Del-
aram Kahrobaei, Vladimir Shpilrain, editors.
AMS Special Session on Algorithmic Problems of Group Theory and Their Complexity, January
9–10, 2013, San Diego, CA.
AMS Special Session on Algorithmic Problems of Group Theory and Applications to Informa-
tion Security, April 6–7, 2013, Boston College, Chestnut Hill, MA.
pages cm. – (Contemporary mathematics ; volume 633)
Includes bibliographical references.
ISBN 978-0-8218-9859-8 (alk. paper)
1. Group theory–Congresses. 2. Noncummutative algebras–Congresses. 3. Algorithms–Con-
gresses. 4. Cryptography–Congresses. 5. Data encryption (Computer science)–Congresses.
6. Algebra–Congresses. I. Kahrobaei, Delaram, 1975– editor. II. Shpilrain, Vladimir, 1960–
editor.
QA176.A454 2014
652.8015122–dc23 2014029814
Contemporary Mathematics ISSN: 0271-4132 (print); ISSN: 1098-3627 (online)
DOI: http://dx.doi.org/10.1090/conm/633
Copying and reprinting. Individual readers of this publication, and nonprofit libraries
acting for them, are permitted to make fair use of the material, such as to copy select pages for
use in teaching or research. Permission is granted to quote brief passages from this publication in
reviews, provided the customary acknowledgment of the source is given.
Republication, systematic copying, or multiple reproduction of any material in this publication
is permitted only under license from the American Mathematical Society. Permissions to reuse
portions of AMS publication content are handled by Copyright Clearance Center’s RightsLink
service. For more information, please visit: http://www.ams.org/rightslink.
Send requests for translation rights and licensed reprints to reprint-permission@ams.org.
Excluded from these provisions is material for which the author holds copyright. In such cases,
requests for permission to reuse or reprint material should be addressed directly to the author(s).
Copyright ownership is indicated on the copyright page, or on the lower right-hand corner of the
first page of each article within proceedings volumes.

c 2015 by the American Mathematical Society. All rights reserved.
The American Mathematical Society retains all rights
except those granted to the United States Government.
Copyright of individual articles may revert to the public domain 28 years
after publication. Contact the AMS for copyright status of individual articles.
Printed in the United States of America.

∞ The paper used in this book is acid-free and falls within the guidelines
established to ensure permanence and durability.
Visit the AMS home page at http://www.ams.org/
10 9 8 7 6 5 4 3 2 1 20 19 18 17 16 15
Contents
Preface vii
Secret sharing using non-commutative groups and the shortlex order
Bren Cavallo and Delaram Kahrobaei 1
An algorithm that decides conjugacy in a certain generalized free product
Anthony E. Clement 9
Classification of automorphic conjugacy classes in the free group on two
generators
Bobbe Cooper and Eric Rowland 13
On elementary free groups
Benjamin Fine, Anthony Gaglione, Gerhard Rosenberger,
and Dennis Spellman 41
An application of a localized version of an axiom of Ian Chiswell
Anthony M. Gaglione, Seymour Lipschutz,
A note on Stallings’ pregroups
Anthony M. Gaglione, Seymour Lipschutz,
A CCA secure cryptosystem using matrices over group rings
Delaram Kahrobaei, Charalambos Koupparis,
and Vladimir Shpilrain 73
The MOR cryptosystem and finite p-groups
Ayan Mahalanobis 81
A group theoretical ElGamal cryptosystem based on a semidirect product of
groups and a proposal for a signature protocol
Anja I. S. Moldenhauer 97
On some algorithmic properties of finite state automorphisms of rooted trees
Benjamin Steinberg 115
v
Preface
This volume consists of contributions by participants and speakers in special

sessions at two AMS meetings. These special sessions concerned algorithmic prob-
lems of group theory, their complexity, and applications to cryptography. The AMS
Special Session on Algorithmic Problems of Group Theory and Their Complexity
was held at the San Diego Convention Center in January 2013 and the AMS Special
Session on Algorithmic Problems of Group Theory and Applications to Information
Security was held at Boston College in April 2013.
Over the past few years the field of group-based cryptography has attracted the
attention of both group theorists and cryptographers. The new techniques inspired
by algorithmic problems in non-commutative group theory and their complexity
have offered promising ideas for developing new cryptographic primitives. This
volume contains both survey and research papers on algorithmic group theory and
applications to cryptography.
We are grateful to the American Mathematical Society for their help in the
publication of this volume. In particular we thank Christine Thivierge for her
patience and assistance in putting this volume together.
Delaram Kahrobaei
Vladimir Shpilrain
vii
Contemporary Mathematics
Volume 633, 2015
http://dx.doi.org/10.1090/conm/633/12646
Secret sharing using non-commutative groups

and the shortlex order
Bren Cavallo and Delaram Kahrobaei

Abstract. In this paper we review the Habeeb-Kahrobaei-Shpilrain secret
sharing scheme and introduce a variation based on the shortlex order on a free
group. Drawing inspiration from adjustments to classical schemes, we also
present a method that allows for the protocol to remain secure after multiple
secrets are shared.
1. Introduction
Secret sharing is a cryptographic protocol by which a dealer distributes a secret
via shares to participants such that only certain subsets of participants can together
use their shares to recover the secret. A secret sharing scheme begins with a dealer,
a secret, participants, and an access structure. The access structure determines
which groups of participants have access to the secret. The goal of the scheme is to
distribute the secret to the participants in such a way that only sets of participants
within the access structure have access to the secret. In this way, it is most often
the case that no individual participant can recover the secret on their own.
Secret sharing schemes are ideal tools for when the secret is both highly impor-
tant and highly sensitive. The fact that there are multiple shares, as opposed to one
private key in private key cryptography, makes the secret less likely to be lost while
allowing high levels of confidentiality. If any one share is compromised the secret
can generally still be recovered with the non-compromised shares. Additionally,
even though the secret is spread out over multiple shares, recovering the secret is
limited by the access structure, and so the secret remains secure. Secret sharing
has applications in multi-party encryption, Byzantine agreement, and threshold en-
cryption among others. See [1] for a survey on secret sharing and its applications
in cryptography and computer science.
2. Formal Definition
A secret sharing scheme consists of a dealer, n participants, P1 , . . . Pn , and an
access structure A ⊆ 2{P1 ,··· ,Pn } such that for all A ∈ A and A ⊆ B, B ∈ A.
To share a secret s, the dealer runs an algorithm:
Share(s) = (s1 , · · · , sn )
and then distributes each share si to Pi .
2010 Mathematics Subject Classification. Primary 20F05, 94A60, 20F10.
2015
c American Mathematical Society
1
2 BREN CAVALLO AND DELARAM KAHROBAEI
In order to recover the secret, participants can run the algorithm Recover which
has the property that for all A ∈ A:
Recover({si : i ∈ A}) = s
and if A ∈/ A then running Recover is either computationally infeasible or impossi-
ble.
As such, only groups of participants in A can access the secret. The monotonic-
ity of A is also apparent in that if A ∈ A and A ⊆ B then the set of participants
in A could also recover the secret for B. A secret sharing scheme is called perfect
if ∀A ∈/ A the shares si ∈ A together give no information about s.
3. Shamir’s Secret Sharing Scheme

One of the more common access structures one sees in secret sharing is the
(k,n) threshold:
A = {A ∈ 2{P1 ,··· ,Pn } : |A| ≥ k}.
Namely, A consists of all subsets of the n participants of size k or greater. We call
a secret sharing scheme that has A as a (k, n) threshold a (k, n) threshold scheme.
The problem of discovering a perfect (k, n) threshold scheme was solved indepen-
dently by G. Blakely [2] and A. Shamir [15] in 1979.
In the Shamir Secret Sharing Scheme, the secret is an element in Zp where p
is a prime number larger than the number of participants. Given a secret s, the
dealer generates the shares for a (k, n) threshold by doing the following:
• The dealer randomly selects a1 , · · · , ak−1 ∈ Zp such that ak−1 = 0 and
constructs the polynomial f (x) = ak−1 xk−1 + · · · + a1 x + s
• For each participant Pi the dealer publishes a corresponding xi ∈ Zp .
The dealer then distributes the share si = f (xi ) to each Pi over a private
channel.
Any subset of k participants can then reconstruct the polynomial f (x) by using
polynomial interpolation and then finding f (0) = s. This method finds s uniquely as
any degree k − 1 polynomial is uniquely determined by the k shares. The shares are
consistent because each (xi , f (xi )) is a point on the polynomial f (x) and thus any k
shares will reconstruct the same polynomial. In order to reconstruct a polynomial
f (x) = a0 + a1 x + · · · ak−1 xk−1 given points (x1 , f (x1 )), · · · , (xk , f (xk )) one can
solve for the coefficients column in the following system of linear equations:
⎛ k−1 ⎞⎛ ⎞ ⎛ ⎞
x1 · · · x1 1 ak−1 f (x1 )
⎜x2 k−1 · · · x2 1⎟ ⎜ak−2 ⎟ ⎜f (x2 )⎟
⎜ ⎟⎜ ⎟ ⎜ ⎟
⎜ .. .. .. .. ⎟ ⎜ .. ⎟ = ⎜ .. ⎟ .
⎝ . . . . ⎠ ⎝ . ⎠ ⎝ . ⎠
xk k−1
· · · xk 1 a0 f (xk )
The above method of interpolation demonstrates that Shamir’s scheme is perfect.
If there were less than k shares, than the system of equations above would have
more equations than unknowns, and there would not be a unique solution for a0 .
4. Secret Sharing Using Non-commutative Groups

Given a set of letters X = {x1 , x2 , . . . , xn } we define the free group generated
by X, F (X), as the set of reduced words in the alphabet X ±1 = {x±1 ±1
1 , . . . , xn },
−1 −1
where a word is reduced if there are no subwords of the form xi xi or xi xi . Given
SECRET SHARING USING NC GROUPS AND THE SHORTLEX ORDER 3
a set of words R ⊂ F (X) we define R
as the smallest normal subgroup of F (X)

containing R and define the group G = X|R
= F (X)/ R
. We call R the set of

relators of G.
A group G = X|R
has a solvable word problem if there exists an algorithm to
determine if any word w ∈ G is trivial. Habeeb-Kahrobaei-Shpilrain (HKS) secret
sharing [7] uses a group with an efficiently solvable word problem to create an
(n, n) threshold scheme which can be extended to a (k, n) threshold scheme using
the method of Shamir.
4.1. (n, n) Threshold. In this case the secret, s, is an element of {0, 1}k which
we view as a column vector. The setting is initialized by making a set of generators
X = {x1 , · · · , xn } public. To distribute the shares the dealer does the following:
• Distributes to each Pi over a private channel a set of words Ri in the
alphabet X ±1 that define the group Gi = X|Ri
.
• Randomly generates the shares si ∈ {0, 1}k for i = 1, · · · , n − 1 and
n−1
sn = s − j=0 sj where the addition is bitwise addition in Fk2 .
• Publishes words wji over the alphabet X ±1 such that a word wji is trivial
in Gi if sji = 1 and non-trivial if sji = 0.
Since the Gi have efficiently solvable word problem, the participant Pk can deter-
mine which of the wjk are trivial or non-trivial and can independently recover sk .
To recover the secret, the Pi add the si and find s. Note that even though the wji
are sent over an open channel, the shares remain secure since the Ri are private.
Therefore no other participant can recover si from the wji since only Pi knows Gi .
4.2. (k, n) Threshold. One can extend the above scheme to a (k, n) threshold
via Shamir’s scheme. As is the case with Shamir’s scheme, the secret s is an element
of Zp and the shares, si , correspond to points on a polynomial of degree k − 1 with
constant term s. The shares are distributed and reconstructed in an identical
manner as above by viewing the si in their binary form. The trivial and non-
trivial words are sent to each Pi so that they reconstruct each si in its binary form.
After recovering their shares any element of the access structure can use polynomial
interpolation to find s:
• The dealer randomly selects a1 , · · · , ak−1 ∈ Zp such that ak−1 = 0 and
constructs the polynomial f (x) = ak−1 xk−1 + · · · + a1 x + s.
• For each participant Pi the dealer publishes a corresponding xi ∈ Zp . The
dealer then converts each si = f (xi ) into binary. And thus, each si can
be viewed as a column vector of length l = log2 p + 1.
• As was the case in the (n, n) scheme, the dealer distributes the si over an
open channel by sending each Pi the words w1i , · · · , wli over the alphabet
X ± such that wji is trivial in Gi if sji = 1 and non-trivial if sji = 0.
• The participants reconstruct their own si and can recover the secret using
polynomial interpolation.
Some advantages this secret sharing scheme has over Shamir’s scheme include the
fact that after the Ri are distributed, one can still use them to send out and
reconstruct more secrets rather than having to privately distribute new shares each
time a different secret is picked. Private information has to only be sent once
initially for an arbitrary amount of secrets to be shared due to the method of
distributing the shares. Despite this, the scheme is vulnerable to an adversary
determining the relators by seeing patterns in words they learn are trivial. Namely,
after a participant reveals their share (possibly while recovering the secret) an
adversary could determine which of the wji were trivial and potentially find the
group presentation of Gi which would allow them to reconstruct Pi ’s share on
their own. As in [7], we assume that this is a computationally difficult problem.
Moreover, in Section 5 we provide a method to update relators over time thus
limiting the amount of information an adversary could obtain about a single group.
Another advantage to this scheme is that since it is based on the Shamir secret
sharing protocol it can benefit from the large amount of research done on Shamir’s
scheme. For instance, the verification methods or proactive secret sharing protocols
from [16] and [8] can still be used in this scheme.
4.3. Small Cancellation Groups. In this section we introduce a candidate

group for the above secret sharing scheme.
A word w is cyclically reduced if it is reduced in all of its cyclic permutations.
Note that this only occurs if the word is freely reduced, it has no subwords of the
form x−1 −1
i xi or xi xi , and the first and last letters are not inverses of each other.
A set of words R is called symmetrized if each word is cyclically reduced and
the entire set and their inverses are closed under cyclic permutation. If R is viewed
as a set of relators, symmetrizing R does not change the resulting group as the
closure R under cyclic permutations and inverses is a subset of the normal closure.
Given a set R we say that v is a piece if it is a maximal initial subword of two
different words, namely if there exist w1 , w2 ∈ R such that w1 = vr1 and w2 = vr2 .
A group G = X|R
satisfies the small cancellation condition C (λ) for 0 < λ < 1
if for all r ∈ R such that r = vw where v is a piece, then |v| < λ|r|.
Small cancellation groups satisfying C ( 61 ) have a linear time algorithm for
the word problem [3] making them an ideal candidate for the HKS secret sharing
scheme. Moreover, it can be seen from their definition that if the number of gen-
erators is large compared to the number of relators and lengths of the relators, it
is likely that there will be small cancellation since the probability that any two
words have a large maximal initial segment is low. After generating a random set
of relators satisfying the above properties, it is also fast to symmetrize the set and
then find the pieces and check that they are no larger than one sixth of the word.
As such, it is fast to create such groups by repeatedly randomly generating relators,
symmetrizing, and checking to see if they satisfy the C ( 16 ) condition. There are
other groups that have an efficient word problem that could also function as can-
didate groups, but small cancellation groups have the advantage of being efficient
to generate randomly.
4.4. Secret Sharing and the Shortlex Ordering. Let X = {x1 , · · · , xn }

and G = X
. A shortlex ordering on G is induced by an order on X ±1 as follows.
Given reduced w = xi1 · · · xip and l = xj1 · · · xjk with w = l then w < l if and only
if:
• |w| < |l|,
• or if p = k and xia < xja where a = minα {xiα = xjα }.
For example, let X = {x, y} and give X ± the ordering x < x−1 < y < y −1 . Then
some of the first words in order would be:
e < x < x−1 < y < y −1 < x2 < xy < xy −1 < x−2 < x−1 y < x−1 y −1 < yx <
yx < y 2 < y −1 x < y −1 x−1 < y −2 < x3 < x2 y < x2 y −1 < xyx < xyx−1 < · · ·
−1
Utilizing the the shortlex ordering, we can modify the HKS (k, n) threshold as
follows:
• The dealer publishes the letters X and over a private channel sends a set
of words, Ri in X ±1 to each Pi such that Gi = X|Ri
is a group with
an efficient algorithm to reduce words with respect to the Ri or compute
normal forms.
• The dealer chooses a secret s ∈ Zp for some large prime p > n and
generates a random polynomial, f in Zp [x] with constant term s.
• The dealer assigns a public xi ∈ Zp to each participant, computes f (xi ),
and finds si ∈ F (X) such that si is the f (xi )th word in F (X). Note that
xi is not a generator of G, but rather the x-coordinate associated to each
participant’s share.
• The dealer publishes a word wi that reduces to si in Gi . This can be
done efficiently by interspersing conjugated products of relators between
the letters of si .
• Each participant Pi computes their share by reducing wi to get si and
then computing its position in F (X).
• Using their shares they find the secret using polynomial interpolation.
The main advantage of this new method is that participants need only reduce
one word rather than a number of words corresponding to the length of the secret.
In general, being able to reduce words is more general than being able to solve the
word problem in a finitely presented group and in some cases may be more complex.
It is important to note the following about this scheme:
• Given an algorithm that reduces words, each wi must reduce uniquely to
si . This implies that if our reduction algorithm does not terminate at si ,
then it is not a viable share for this scheme. In that case, if a random f (xi )
does not correspond to a fully reduced word or a word in normal form,
the dealer can always assign Pi a different xi . It may also be necessary to
check that each wi reduces to si give the reduction algorithm before the
shares are distributed.
• Some reduction algorithms can be done in multiple ways given the same
initial conditions and can terminate at different words. As such, it is
important to fix a protocol so that whatever process Pi uses to reduce wi
terminates at si .
4.5. Platform Group. For this variant of the HKS secret sharing scheme,
we also propose C ( 16 ) groups. Additionally, we propose the parameters |X| =
40, |R| = 4, and |r| = 9 for all r ∈ R. We find that with such parameters,
generating a single C ( 61 ) group can be done in roughly 1 second in GAP [6] by
generating random relators of the given length and then checking that the set of
relators satisfies the small cancellation condition. In order to reduce the wi to si ,
participants can use Dehn’s algorithm which terminates in linear time [3]. It is not
guaranteed in general that Dehn’s algorithm will reduce each wi to si , as such it
is necessary to check that each wi reduces to si . In order to test the efficacy of
Dehn’s algorithm in C ( 16 ) groups for the purposes of this secret sharing scheme,
we performed the following tests in GAP [6]:
• Generate 10 small cancellation groups using the parameters from the first
paragraph of this section.
• In each group we generated 100 words of length less than 10 and created
corresponding large unreduced words of length 500 by inserting conju-
gated products of relators between letters in our original word.
• Applied an implementation of Dehn’s algorithm due to Chris Staecker [17]
and checked that our unreduced word successfully reduced to the original
word.
After running said tests, we found that Dehn’s algorithm successfully reduced every
word. The size considerations in the second item were given in part because there
are enough non-trivial, Dehn reduced, words of length 10 or less in the free group
on 40 generators to be used as shares in a practical setting.
4.6. Efficiency. Each step in modified HKS scheme can be done efficiently.
As mentioned previously, generating C ( 16 ) groups can be done quickly by repeat-
edly generating sets of relators and checking to see if they satisfy the necessary
small cancellation condition. The necessary computations using the shortlex order-
ing can be done using basic combinatorial formulas that are very fast for a computer
to evaluate. Additionally, the wi can be created efficiently from the si by inserting
conjugated products of relators and then reduced in polynomial time using Dehn’s
algorithm. Moreover, the dealer can also check that the wi reduce to the si effi-
ciently. Hence each additional step to the standard Shamir’s scheme can be done
efficiently. This is also an improvement over the standard HKS scheme since the
amount of words that need to be reduced is independent of the length of the secret,
making it possible for larger secrets to be distributed efficiently.
5. Updating Relators
The main security concern for this cryptoscheme is the possibility of an ad-
versary discovering a participant’s set of relators. This can either be done using
information gained from combining shares, but even potentially just from the pub-
lic wi . As more secrets are shared, the original set of relators becomes less secure.
Moreover, information may be discovered either by breaking into wherever a par-
ticipant stores their relators or if partial information was discovered during the
initial step. In this section we present a method to refresh a participant’s relator
set using the same inherent security assumptions necessary for the cryptoscheme,
namely that at least one round of secret sharing is secure. To do this we add steps
that can take place before any new secret is sent out:
• For each Pi the dealer creates a set of words, Ri , in X ±1 such that Gi =
X|Ri
satisfies the same desired properties.
• In order to distribute each r ∈ Ri , the dealer pads r with relators in Ri
as done previously and publishes them.
• Pi then reduces r by using the relators in Ri .
• After the full set of words in Ri is published and reduced, Pi deletes the
original Ri and sets Ri := Ri .
If these steps are done before an adversary can gain adequate information about
relators, then after an update phase the information an adversary has gained will
be largely rendered useless. Also note that a single secret can be kept secure over
a long period of time using the methods in [8]. In this case, it is important that
the words in Ri are reduced with respect to the original Ri . As such, Ri and Ri
are not completely unrelated, but as the relators become updated each additional
time, they will have less and less to do with the original set of relators.
6. Conclusion
In this paper we propose a modification of the HKS secret sharing scheme
using the shortlex ordering on free groups. It improves the original scheme by
removing the relation of the number of times each participant has to solve the word
problem to the length of the secret. As such, larger secrets can be shared efficiently
and the overall scheme is more efficient. Moreover, it shares the advantage over
Shamir’s scheme that multiple secrets can be shared given the same initial private
information. We also introduce a method to update relators so that the scheme
remains secure when arbitrarily many secrets are shared and that does not involve
more private information being distributed.
Support. Delaram Kahrobaei was partially supported by the Office of Naval

Research grant N00014120758 and also supported by PSC-CUNY grant from the
CUNY research foundation, as well as the City Tech foundation.
References
[1] Amos Beimel, Secret-sharing schemes: a survey, Coding and cryptology, Lecture Notes in
Comput. Sci., vol. 6639, Springer, Heidelberg, 2011, pp. 11–46, DOI 10.1007/978-3-642-20901-
7 2. MR2834691 (2012h:94185)
[2] G.R. Blakley. Safeguarding cryptographic keys. In Proceedings of the 1979 AFIPS National
Computer Conference, pages 313–317, Monval, NJ, USA. AFIPS Press.
[3] B. Domanski and M. Anshel, The complexity of Dehn’s algorithm for word problems in groups,
J. Algorithms 6 (1985), no. 4, 543–549, DOI 10.1016/0196-6774(85)90031-8. MR813591
(87e:20066)
[4] J. Feigenbaum (ed.), Advances in cryptology—CRYPTO ’91, Lecture Notes in Computer
Science, vol. 576, Springer-Verlag, Berlin, 1992. MR1243642 (94e:94001)
[5] Paul Feldman, A practical scheme for non-interactive verifiable secret sharing. In Proceedings
of the 28th Annual Symposium on Foundations of Computer Science, SFCS ’87, pages 427–
438, Washington, DC, USA, 1987. IEEE Computer Society.
[6] The GAP Group, GAP – Groups, Algorithms, and Programming, Version 4.7.6, 2014.
http://www.gap-system.org.
[7] Maggie Habeeb, Delaram Kahrobaei, and Vladimir Shpilrain, A secret sharing scheme based
on group presentations and the word problem, Computational and combinatorial group the-
ory and cryptography, Contemp. Math., vol. 582, Amer. Math. Soc., Providence, RI, 2012,
pp. 143–150, DOI 10.1090/conm/582/11557. MR2987392
[8] Amir Herzberg, Markus Jakobsson, Stanisllaw Jarecki, Hugo Krawczyk, and Moti Yung,
Proactive public key and signature systems. In Proceedings of the 4th ACM conference on
Computer and communications security, CCS ’97, pages 100–110, New York, NY, USA, 1997.
ACM.
[9] Derek F. Holt, Bettina Eick, and Eamonn A. O’Brien, Handbook of computational group
theory, Discrete Mathematics and its Applications (Boca Raton), Chapman & Hall/CRC,
Boca Raton, FL, 2005. MR2129747 (2006f:20001)
[10] S.M. Jarecki, Proactive Secret Sharing and Public Key Cryptosystems, Massachusetts Insti-
tute of Technology, Department of Electrical Engineering and Computer Science, 1996.
[11] Jonathan Katz and Yehuda Lindell, Introduction to modern cryptography, Chapman &
Hall/CRC Cryptography and Network Security, Chapman & Hall/CRC, Boca Raton, FL,
2008. MR2371431 (2009b:94051)
[12] Ueli Maurer (ed.), Advances in cryptology—EUROCRYPT ’96, Lecture Notes in Computer
Science, vol. 1070, Springer-Verlag, Berlin, 1996. MR1421576 (97g:94002)
[13] Alexei Myasnikov, Vladimir Shpilrain, and Alexander Ushakov, Group-based cryptogra-
phy, Advanced Courses in Mathematics. CRM Barcelona, Birkhäuser Verlag, Basel, 2008.
MR2437984 (2009d:94098)
[14] Torben Pryds Pedersen, Noninteractive and information-theoretic secure verifiable secret
sharing, Advances in cryptology—CRYPTO ’91 (Santa Barbara, CA, 1991), Lecture Notes
in Comput. Sci., vol. 576, Springer, Berlin, 1992, pp. 129–140, DOI 10.1007/3-540-46766-1 9.
MR1243648
[15] Adi Shamir, How to share a secret, Comm. ACM 22 (1979), no. 11, 612–613, DOI
10.1145/359168.359176. MR549252 (80g:94070)
[16] Markus Stadler, Publicly verifiable secret sharing, Advances in cryptology—EUROCRYPT
’96 (Ueli Maurer, ed.), Lecture Notes in Computer Science, vol. 1070, Springer-Verlag, Berlin,
(1996) pp. 190–199.
[17] Chris Staecker, dehn.gap, http://cstaecker.fairfield.edu/˜cstaecker/files/gap/dehn.gap.
CUNY Graduate Center, City University of New York

E-mail address: bcavallo@gc.cuny.edu
CUNY Graduate Center and City Tech, City University of New York
E-mail address: dkahrobaei@gc.cuny.edu
Volume 633, 2015
An algorithm that decides conjugacy

in a certain generalized free product
Anthony E. Clement
Abstract. Inspired by G. Baumslag’s paper “On generalized free products”,
we consider a certain type of cyclically pinched generalized free product G
which is residually free. We devise an explicit algorithm to solve the conjugacy
problem in G.
1. Introduction
Although the word problem is algorithmically solvable for generalized free prod-
ucts of finitely generated free groups when the amalgamated subgroups are finitely
generated, the conjugacy problem is more elusive even for these groups. Many con-
tributions have been made in this area. In [4], for example, S. Lipschutz states that
if A and B are two groups, each with solvable conjugacy problem, then the free
product of A and B with cyclic amalgam again has solvable conjugacy problem,
provided that the generators of the cyclic groups being amalgamated satisfy certain
criterion. Unfortunately, no explicit algorithm is provided.
In this paper, an explicit algorithm is given that decides conjugacy for a certain
residually free generalized free product. More specifically, take a free group F =
x, y
of rank 2, a non-trivial element u in F that generates its own centralizer in
F , and a free abelian group A of rank 2 with a set {t, v} of independent generators.
∗ A with cyclic amalgam. By a theorem
Form the generalized free product G = Fu=v
of G. Baumslag [1], G is residually free. We devise an explicit algorithm for solving
the conjugacy problem in G.
2. Preliminaries
Our algorithm deals with the conjugacy problem in a free product with amal-
gamation which is residually free. We rely on a classical result which describes a
property about cyclically reduced elements of any free product with amalgamation.
Theorem 2.1 ([5]). Let G = AH=K ∗ B . Then every element of G is conjugate to
a cyclically reduced element of G. Moreover, suppose that g is a cyclically reduced
element of G. Then:
(i) If g is conjugate to an element h in H, then g is in some factor and there is a
sequence h, h1 , h2 , ..., ht , g where hi is in H and consecutive terms of the sequence
are conjugate in a factor.
2010 Mathematics Subject Classification. Primary 20E06; Secondary 20F10.

Key words and phrases. Algorithm, conjugacy, generalized free product.
9 2015
10 ANTHONY E. CLEMENT
(ii) If g is conjugate to an element g in some factor, but not in a conjugate of H,

then g and g are in the same factor and are conjugate in that factor.
(iii) If g is conjugate to an element p1 · · · pr , where r ≥ 2, and pi , pi+1 as well as
p1 , pr are in distinct factors, then g can be obtained by cyclically permuting p1 · · · pr
and then conjugating by an element of H.
We will make use of three well-known results which pertain to algorithm prob-
lems.
Proposition 2.2 ([4] The Generalized Word Problem). Given a finite subset
U of a finitely generated free group F , there is an algorithm which decides whether
or not elements in F are in gp(U ).
Proposition 2.3 ([4]). Given a finite subset {u, p1 , g1 } in a free group F , there
is an algorithm that decides whether or not there exist integers i and j such that
uj p1 ui = g1 holds in F and if they do, finds them.
Remark : Let H = gp(u, p1 ). If u and p1 do not commute, then H is free on
u and p1 . By Proposition 2.2, we can decide if an element lies in H. If it does,
Proposition 2.3 shows that an expression for it can be found in terms of the given
basis {u, p1 } for H.
Theorem 2.4 ([5]). For any free group or free abelian group, there exists an
algorithm that decides whether or not two given elements are conjugate.
3. The Algorithm
We now construct an algorithm for the problem proposed in the introduction.
Let F = x, y
be a free group of rank 2, and let u be a non-trivial element in F
that generates its own centralizer in F. Let A be a free abelian group A of rank 2
with a basis {t, v}. Set H = u
and K = v
.
Theorem 3.1. There exists an algorithm that decides whether or not two given
elements g and g in G = Fu=v
∗ A are conjugate.
Notation: If g and h are conjugate, we write g ∼ h.
Proof. Let g, g ∈ G. We will use Theorem 2.1 as a blueprint to derive an

algorithm that decides whether or not g ∼ g . By Theorem 2.1, we may as well
assume that g is cyclically reduced. Suppose that g is, indeed, conjugate to g . By
Theorem 2.1, there are three cases to consider. Since g, g ∈ G = F u=v ∗ A , the
first two cases are easily handled by Theorem 2.4. We focus our attention on the
third case. First we deduce the various forms that g and g can have in order for
them to be conjugate. Once this is established, we provide our algorithm.
Suppose g = p1 · · · pr , where r ≥ 2 and pi , pi+1 , as well as p1 , pr , are in distinct
factors. This implies that r is always even. If g = g1 · · · gr , then we can have either
(a) g1 ∈ F , g2 ∈ A, and gr ∈ A, or (b) g1 ∈ A, g2 ∈ F, and gr ∈ F (since the
sequence must be alternating). We consider case (a); case (b) is similar.
Assume g1 ∈ F , g2 ∈ A, and gr ∈ A. We want to obtain a relationship between
gi and pi for i = 1, 2, . . . , r. By uniqueness of the normal form for generalized free
products, there exists h1 ∈ H such that g1 = p1 h1 . Suppose g = g1 · · · gr ∼ g =
p1 · · · pr , then p1 h1 g2 · · · gr ∼ p1 · · · pr . Similarly, for some h2 ∈ H, h1 g2 = p2 h2
ALGORITHM DECIDES CONJUGACY IN A GENERALIZED FREE PRODUCT 11
and, thus, g2 = h−1 −1

1 p2 h2 . Continuing in this manner, we find that gr = hr−1 pr hr
for some hr , h−1
r−1 ∈ H.
Now, Theorem 2.1 (iii) implies that there exists h ∈ H such that one of the
following cases holds:
1) h−1 p1 p2 · · · pr h = g1 g2 · · · gr
or
2) h−1 p3 p4 · · · pr p1 p2 h = g1 g2 · · · gr
or
..
.
or
r
2) h−1 pr−1 pr p1 · · · pr−2 h = g1 g2 · · · gr with r > 2 and r even.
We now have the various types of conjugacy relations between g and g . It will
be useful to rewrite the left hand side of each case by inserting hh−1 between each
pair pi and pi+1 . Thus, Case 1) can be expressed in the form
(h−1 p1 h)(h−1 p2 h) · · · (h−1 pr h) = g1 g2 · · · gr .
We are ready to illustrate how the algorithm works. Consider Case 1). Take
any two elements g = g1 · · · gr and g = p1 · · · pr , say, with g1 ∈ F , g2 ∈ A, p1 ∈ F ,
p2 ∈ A, and gr in A. (Here, we are using the same notation as before. These are not
the same g and g as above, and we are not assuming that g and g are necessarily
conjugate at this point). We wish to decide whether or not h−1 gh = g for some
h ∈ H. By Proposition 2.2, we can to detect whether or not gi lies in gp(u, pi ) for
each i = 1, 2, . . . , r. Due to the normal form for generalized free products and by
Proposition 2.3, for the existence of some h in G to have this property above in
Case 1), we would need to have
(∗) h = ui and u−i p1 ui = g1
in F for some positive integer i. By Proposition 2.2, we would need to have g1 ∈

gp(u, p1 ). Note that if g1 ∈ gp(u, p1 ), then g1 has unique expression of the form
g1 = uα1 pβ1 1 uα2 pβ1 2 · · · uαn pβ1 n for some integers αi and βi . We can compare this
word with (∗) and decide whether or not g1 ∈ gp(u, p1 ). If g1 ∈ gp(u, p1 ), we
check to see if g2 ∈ gp(u, p2 ) keeping this same permutation. If, further, g2 ∈
gp(u, p2 ), · · · , gr ∈ gp(u, pr ) in the first permutation and u−i pk ui = gk (1 ≤ k ≤ r)
for the same i, then g = g1 · · · gr is conjugate to g = p1 · · · pr . We can repeat the
process used in Case 1) to any of the other permutations. If in any permutation we
get consistently affirmative answers, as illustrated in Case 1), then the algorithm
confirms that g ∼ g .
Returning to the illustrative Case 1), if g1 ∈ / gp(u, p1 ), then we go to Case 2)
and repeat the process; that is, we can check to see if g1 ∈ gp(u, p3 ), etc. If we
continue in this way and gk ∈ gp(u, p ) for any pair k and (1 ≤ k, ≤ r), where
gk and p are identified as being in the same matching corresponding factor in the
sequence of the permutation for every permutation going from Case 1) through
12 ANTHONY E. CLEMENT
Case 2r ), or the value of i (the u exponent) is not the same in any permutation from
1 to r2 , then we conclude g g .
The following illustrates a more routine algorithmic approach:

For each cyclic permutation of g1 g2 · · · gr , do the following routine: First decide
if gk ∈ gp(u, p ) for any pair k and (1 ≤ k, ≤ r), where gk and p are identified as
being in the same matching corresponding factor in the sequence of the permutation.
If it is the case that gk ∈ gp(u, p ) for all pairs k and (1 ≤ k, ≤ r), where gk
and p are being identified as being in the same matching corresponding factor in
the sequence of the permutation and u−i p ui = gk for the same value of i in the
permutation, then we conclude that g ∼ g .
If gk ∈ gp(u, p ) for any pair k and (1 ≤ k, ≤ r), where gk and p are
identified as being in the same matching corresponding factor for each permutation
going from Case 1) through Case r2 ) or the value of i (the u exponent) is not the
same in any permutation going from Case 1) through Case r2 ), then we conclude
that g g .
The final decision will be reached by the r2 th running of the process.
References
[1] Gilbert Baumslag, On generalised free products, Math. Z. 78 (1962), 423–438. MR0140562
(25 #3980)
[2] Anthony E. Clement, On the Baumslag-Solitar groups and certain generalized free prod-
ucts, ProQuest LLC, Ann Arbor, MI, 2006. Thesis (Ph.D.)–City University of New York.
MR2709227
[3] Seymour Lipschutz, The conjugacy problem and cyclic amalgamations, Bull. Amer. Math. Soc.
81 (1975), 114–116. MR0379675 (52 #580)
[4] Roger C. Lyndon and Paul E. Schupp, Combinatorial group theory, Springer-Verlag, Berlin-
New York, 1977. Ergebnisse der Mathematik und ihrer Grenzgebiete, Band 89. MR0577064
(58 #28182)
[5] Wilhelm Magnus, Abraham Karrass, and Donald Solitar, Combinatorial group theory: Pre-
sentations of groups in terms of generators and relations, Interscience Publishers [John Wiley
& Sons, Inc.], New York-London-Sydney, 1966. MR0207802 (34 #7617)
Department of Mathematics, Brooklyn College, City University of New York,

Brooklyn, New York
E-mail address: aclement@brooklyn.cuny.edu
Volume 633, 2015
Classification of automorphic conjugacy classes

in the free group on two generators
Bobbe Cooper and Eric Rowland
Abstract. We associate a finite directed graph with each equivalence class

of words in F2 under AutF2 , and we completely classify these graphs, giving
a structural classification of the automorphic conjugacy classes of F2 . This
classification refines work of Khan and proves a conjecture of Myasnikov and
Shpilrain on the number of minimal words in an automorphic conjugacy class
whose minimal words have length n, which in turn implies a sharp upper
bound on the running time of Whitehead’s algorithm for determining whether
two words in F2 are automorphic conjugates.
1. Introduction
We begin with a few standard definitions. Let F2 = a, b
be the free group on
two generators a and b. The length of w ∈ F2 is denoted by |w|. A word w ∈ F2 is
minimal if |φ(w)| ≥ |w| for all φ ∈ Aut F2 .
Two elements w and v in F2 are automorphic conjugates if there is an automor-
phism φ ∈ Aut F2 such that φ(w) = v. We write w ∼ v if w and v are automorphic
conjugates. Equivalence classes under ∼, which we refer to as automorphic conju-
gacy classes, are the main object of study in this paper.
An automorphic conjugacy class W supports a natural graph structure in which
the vertices are the words in W and a directed edge is drawn from w to v for
each automorphism φ such that φ(w) = v. Here we will be interested in the
subgraph consisting of minimal words, say of length n, and in particular we will
define (in Section 2) a quotient Γ(W ) of this subgraph obtained by dividing by n
inner automorphisms and 8 permutations.
The size of Γ(W ) has implications for the running time of a standard algorithm
for determining whether two words in F2 are automorphic conjugates. To bound
the time complexity of this algorithm, Myasnikov and Shpilrain [5] studied the
number of minimal words in an automorphic conjugacy class W . They showed that
if w ∈ F2 is a minimal word of length n, then the number of minimal words in its
automorphic conjugacy class is bounded above by a polynomial in n. Further, they
conjectured that 8n2 − 40n gives a sharp bound for n ≥ 9. In terms of Γ(W ), where
we have divided by 8n automorphisms, this is equivalent to the statement that
|V (Γ(W ))| ≤ n − 5 for n ≥ 9. Khan [3] showed that this conjectured bound holds
2010 Mathematics Subject Classification. Primary 20E36, Secondary 68R15.
2015
13
14 BOBBE COOPER AND ERIC ROWLAND
for sufficiently large classes. His approach was to identify a number of subgraphs
that Γ(W ) avoids and use these subgraphs to bound the number of vertices.
Theorem (Khan). If W is an automorphic conjugacy class of size |V (Γ(W ))|
≥ 4373 whose minimal words have length n ≥ 10, then |V (Γ(W ))| ≤ n − 5.
In this paper we take a direct approach to analyzing the structure of Γ(W ).
We are able to recast Khan’s results with shorter proofs and additional information
sufficient to prove the conjecture of Myasnikov and Shpilrain.
Theorem 1.1. If W is an automorphic conjugacy class whose minimal words
have length n ≥ 9, then |V (Γ(W ))| ≤ n − 5.
Myasnikov and Shpilrain [5] perceived the possibility of a sharp polynomial
bound as quite surprising. We show in this paper that the structure of automor-
phic conjugacy classes is quite restricted, perhaps much more so than previously
suspected, which accounts for a simple bound.
Our work builds on that of a previous paper [1] in which we identified certain
words in F2 as root words. We define these words below, following Theorem 1.6. The
property of being a root word is respected by automorphic conjugacy (Theorem 1.8
below), so each automorphic conjugacy class W can be said to either be a root class
or a non-root class. For graphs of sufficiently large automorphic conjugacy classes,
Khan [3] also identified a dichotomy — either the number of vertices is bounded
by some absolute constant or the graph has at most n − 5 vertices and simple edge
structure. We show in this paper that the former correspond to root classes and
the latter to non-root classes.
Both Khan’s approach and ours are founded on a theorem of Whitehead [6, 7]
which provides a finite set of generators for Aut F2 . Before recalling this theorem
we introduce a bit of notation. Let L2 = {a, b, a−1 , b−1 }. For x ∈ L2 , denote
x = x−1 . We identify each element w ∈ F2 with its word on the alphabet L2 in
which no pair of adjacent letters are inverses of each other.
A Type I automorphism or a permutation is an automorphism which permutes
L2 . There are 8 permutations.
Type II automorphisms are defined as follows. Let x ∈ L2 and A ⊂ L2 \ {x, x}.
Define a map φ : L2 → F2 by
φ(y) = xβ(y∈A) y xβ(y∈A) ,
where β(true) = 1 and β(false) = 0. Since φ(y)−1 = φ(y) for all y ∈ L2 , this
map extends to an automorphism. We write φ = (A, x) and call φ a Type II
automorphism. For example, the automorphism φ = ({a}, b) maps a → ab and a →
ba and leaves b, b fixed. This notation for Type II automorphisms was introduced
by Higgins and Lyndon [2]; see also the standard book of Lyndon and Schupp [4,
page 31].
Theorem (Whitehead). If w, v ∈ F2 such that w ∼ v and v is minimal, then
there exists a sequence φ1 , φ2 , . . . , φm of Type I and Type II automorphisms such
that
• φm · · · φ2 φ1 (w) = v and
• for 0 ≤ k ≤ m − 1, |φk+1 φk · · · φ2 φ1 (w)| ≤ |φk · · · φ2 φ1 (w)|, with strict
inequality unless φk · · · φ2 φ1 (w) is minimal.
CLASSIFICATION OF AUTOMORPHIC CONJUGACY CLASSES IN F2 15
To determine whether a word w is minimal, by Whitehead’s theorem it suffices

to apply each Type II automorphism to w. Then w is minimal if and only if
|φ(w)| ≥ |w| for each Type II automorphism φ.
In fact we do not need to check all Type II automorphisms to determine mini-
mality. For example, ({}, x) is the identity automorphism, so we may require that
no automorphism φi in Whitehead’s theorem is ({}, x).
Additionally, notice that ({y, y}, x) is an inner automorphism, since it conju-
gates y by x and also (trivially) conjugates x by x. We view inner automorphisms
as “cosmetic” automorphisms, and we will usually dispense with them by dividing
Aut F2 by its normal subgroup Inn F2 . For clarity, however, our notation will indi-
cate when we have omitted an inner automorphism. We write w ≡ v if φ(w) = v for
some inner automorphism φ. Equivalence classes under ≡ are called cyclic words.
Let C2 be the set of words w = x1 · · · xn ∈ F2 such that xn = x1 . Words in C2
are representatives of cyclic words. For the remainder of the paper, all words are
elements of C2 . Since F2 \ C2 consists entirely of words which are not minimal, we
do not lose any structural information regarding minimal words by moving from F2
to C2 .
Since an inner automorphism does not decrease the length of any word in
C2 , by Whitehead’s theorem we need not consider them when determining the
minimality of a word in C2 . Therefore the primary automorphisms of interest are
automorphisms φ = (A, x) where |A| = 1. We call such an automorphism a one-
letter automorphism. For y ∈ / {x, x}, the one-letter automorphism ({y}, x) maps
x → x, x → x, y → yx, and y → xy. The inverse of φ = ({y}, x) is the one-letter
automorphism φ−1 = ({y}, x).
One-letter automorphisms do not commute with permutations in general, but
we have the following identity, which we will use a number of times.
Lemma 1.2. Let y ∈/ {x, x}, let φ = ({y}, x) be a one-letter automorphism, and
let π ∈ Aut F2 be a permutation. Then πφ = ({π(y)}, π(x))π.
Proof. One checks that both sides map x → π(x) and y → π(y)π(x).
We mention that a consequence of Lemma 1.2 is that one can pull any permu-
tations in the product φm · · · φ2 φ1 to the left. Therefore in Whitehead’s theorem
one may assume that φ1 , φ2 , . . . , φm−1 are Type II automorphisms and that φm is
a permutation.
There are 8 one-letter automorphisms; they are given by ({y}, x) as x and y
run over L2 subject to y ∈
/ {x, x}. Each one-letter automorphism ({y}, x) can be
written as the product
(1.1) ({y}, x) = ({y, y}, x)({y}, x)
of an inner automorphism and another one-letter automorphism. That is, we have
({y}, x)(w) ≡ ({y}, x)(w) for all w ∈ C2 . Therefore, there are only four distinct
one-letter automorphisms modulo Inn F2 . The four principal automorphisms are
({a}, b), ({a}, b), ({b}, a), and ({b}, a); they are distinct modulo Inn F2 . We have
shown the following corollary of Whitehead’s theorem.
Corollary 1.3. Let w ∈ C2 . Then w is minimal if and only if none of the
principal automorphisms decrease the length of w.
Example. Let w = aa. Since the lengths of ({a}, b)(w) = abab, ({a}, b)(w) =
abab, ({b}, a)(w) = aa, and ({b}, a)(w) = aa are at least 2, w is minimal.
By counting two-letter subwords of w we can determine whether the length of

({y}, x)(w) is greater than, less than, or equal to |w|. Hence the minimality of w can
be expressed in terms of these subword counts; this is the content of Theorem 1.6
below. Our notation for counting subwords is as follows. If w = x1 · · · xn and u
are nonempty words in C2 such that k = |u| ≤ |w| = n, let (u)w denote the total
number of (possibly overlapping) occurrences of the (contiguous) subwords u and
u−1 in x1 · · · xn x1 · · · xk−1 . If |u| > |w|, let (u)w = 0. Essentially we are considering
w to be a cyclic word; if w ≡ w then (u)w = (u)w .
Example. Let w = aabbababa; the length-2 subword counts are (aa)w = 2,
(bb)w = 1, (ab)w = 1 = (ba)w , and (ab)w = 2 = (ba)w .
One can show that, in general, (xy)w = (yx)w for w ∈ C2 and x, y ∈ L2 .
In the remainder of this section we give some facts from our previous paper [1]
that we will use. We include a proof of the first lemma to indicate the flavor of the
proofs.
Lemma 1.4. Let w ∈ C2 , and let φ = ({y}, x) with y ∈
/ {x, x}. Then
(yy)φ(w) = (yxy)w ,
(xx)φ(w) = (yxy)w + (yxx)w + (xxy)w + (xxx)w .
Proof. The only way that yy can occur in φ(w) is as the image of yxy in
w. Similarly, yy occurs in φ(w) only where yxy occurs in w; this yields the first
equality. The second equality follows from the observation that xx is introduced
in φ(w) where yxy and yxx occur in w, and xx in w is preserved under φ except
when followed by y; similarly for its inverse xx.
An automorphism φ ∈ Aut F2 is level on w ∈ C2 if |w| = |v| for some v ∈ C2
such that v ≡ φ(w). In other words, φ is level on w if the lengths of w and φ(w)
as cyclic words are equal. For example, ({b}, a) is level on abab but is not level on
abab.
The following lemma is a rephrasing of the statement that a one-letter auto-
morphism is level on w precisely when the number of (cyclic) letter cancellations
it causes is equal to the number of additions. (We must exclude words of length
1; since cyclically consecutive as in w = a are not actually distinct, there is an
addition under ({a}, b) that is not captured by counting occurrences of aa.)
Lemma 1.5. Let w ∈ C2 such that |w| ≥ 2, and let y ∈ / {x, x}. Then the
automorphism ({y}, x) is level on w if and only if (yx)w = (yx)w + (yy)w .
The next theorem follows easily from Corollary 1.3 and Lemma 1.5.
Theorem 1.6. A word w ∈ C2 is minimal if and only if
|(ab)w − (ab)w | ≤ min((aa)w , (bb)w ).
Root words are words satisfying the boundary case of this inequality.
Definition. A word w ∈ C2 is a root word if
|(ab)w − (ab)w | = (aa)w = (bb)w .
This definition is different than, but equivalent to, the definition used in our
previous paper [1, Theorem 7].
Examples of root words include abab, aabb, and abab; these words belong to
classes 4.2 and 4.3 in Appendix A, which lists representatives of all classes contain-
ing a word of length n ≤ 9.
Theorem 1.7. If w is a root word, then |w| is divisible by 4.
An automorphic conjugacy class W is a root class if it contains a root word
and a non-root class if it does not. Theorem 1.8 states that all minimal words in a
root class are root words.
Theorem 1.8. If w is a root word, w ∼ v, and |w| = |v|, then v is a root word.
A word w ∈ C2 is alternating if (aa)w = 0 = (bb)w . For example, abab and
abab are alternating.
Theorem 1.9. Let w ∈ C2 . The following are equivalent.
• w is an alternating minimal word.
• w is an alternating root word.
• The four principal one-letter automorphisms are level on w.
The outline of the paper is as follows. The following section contains the
definition of the graph Γ(W ) and the main theorems of the paper. These theorems
are proved in Sections 3 and 4. We conclude in Section 5 with conjectures on the
number of automorphic conjugacy classes whose minimal words have length n.
2. The graph Γ(W )

In this section we define Γ(W ), a directed graph associated with an automorphic
conjugacy class W . We then state Theorems 2.1–2.3, which classify these graphs.
The basic idea is to consider a graph where the vertices are minimal words in
W and an edge from w to v represents a one-letter automorphism that maps w to
v. Note that there are finitely many minimal words in W , since there are finitely
many words of length n. Therefore the vertex set is finite. To reduce the number of
vertices, we only select distinct minimal words up to “cosmetic” similarity. Namely,
if two minimal words are mapped to each other by an inner automorphism and a
permutation, then we consider them to be representatives of the same vertex.
More formally, let J be the subgroup of automorphisms of F2 generated by
inner automorphisms and permutations. Write w ∼J v if φ(w) = v for some φ ∈ J.
In particular, if w ≡ v then w ∼J v. Define [w] to be the equivalence class of
w under ∼J , and let the vertices of Γ(W ) be the equivalence classes of minimal
words in W under ∼J . Note that the vertices in the graphs considered by Khan [3]
are equivalence classes modulo inner automorphisms only; hence his graph for an
automorphic conjugacy class W has up to 8 times as many vertices as Γ(W ) (fewer
if there are symmetries in a word).
We now describe the edges of Γ(W ). Since J is not a normal subgroup of
Aut F2 , we cannot define φ([w]) to be [φ(w)], because the map u → [φ(u)] is not
invariant on the minimal words in [w].
Example. Consider w = aa and v = bb ∈ [w]. Let φ = ({b}, a). We have
φ(w) = w = aa and φ(v) = baba, and it is clear that [aa] = [baba].
Instead, if φ is a one-letter automorphism, let [φ] be the equivalence class of φ
modulo Inn F2 . Let w, v ∈ C2 be minimal words such that w ∼ v. We say that [w]
is connected to [v] by [φ] if φ(w) ∈ [v]. We draw one directed edge in Γ(W ) from
[w] to [v] for each equivalence class [φ] of one-letter automorphisms such that [w]
is connected to [v] by [φ].
To show that Γ(W ) is well-defined, we must show that the number of edges from
[w] to [v] does not depend on the representatives. First we show that the property
of two vertices being connected does not depend on the representatives. Indeed,
suppose that [w] is connected to [v] by [φ], and let w ∈ [w] and v ∈ [v]. Then w ≡
π(w) for some permutation π; letting φ = πφπ −1 gives φ (w ) ≡ πφ(w) ∈ [v] = [v ].
By Lemma 1.2, φ is a one-letter automorphism, so [w ] is connected to [v ] by [φ ].
Note that in general [φ ] = [φ]. However, the map φ → πφπ −1 is a bijection on the
set of one-letter automorphisms. Moreover, one-letter automorphisms which are
equivalent modulo Inn F2 have images under this map that are equivalent modulo
Inn F2 ; this can be seen from Lemma 1.2. Therefore the number of edges from
[w] to [v] is independent of the representatives chosen. Hence the graph Γ(W ) is
well-defined.
By Whitehead’s theorem, Γ(W ) is connected. We see that, by definition, the
outdegree of each vertex in Γ(W ) is at most 4. Note that Γ(W ) can have loops and
multiple edges.
Example. Consider the automorphic conjugacy class W containing the min-
imal word aabb. This class is class 4.3 in Appendix A. The images of aabb under
the principal one-letter automorphisms are
({a}, b)(aabb) = ababbb,
({a}, b)(aabb) = abab,
({b}, a)(aabb) = aababa,
({b}, a)(aabb) ≡ abab.
The first and third images are not minimal, so they are not represented in Γ(W ).
The second and fourth images are elements of [abab], which is distinct from the
vertex [aabb]. So let us compute the images of abab under the principal automor-
phisms:
({a}, b)(abab) = abba,
({a}, b)(abab) = aabb,
({b}, a)(abab), = abab (a loop),
({b}, a)(abab) = abab (a loop).
The first two images are elements of [aabb], so |V (Γ(W ))| = 2 and Γ(W ) is
+.
[aabb] kn [abab] .
G
The words listed in Appendix A for each automorphic conjugacy class are rep-
resentatives of the vertices of Γ(W ). They are the minimal words in W that appear
first lexicographically (with the order a < b < a < b on L2 ) among their images
under inner automorphisms and permutations. From the listed representatives, one
can compute Γ(W ) by drawing an edge from [w] to [v] for each principal automor-
phism φ such that φ(w) ∼J v.
If there is an edge in Γ(W ) from [w] to [v] then there is an edge from [v] to [w],
since if φ(w) = v then φ−1 (v) = w. Therefore we say that [w] and [v] are neighbors
if there is an edge from [w] to [v] (and from [v] to [w]) without distinguishing
“out-neighbors” from “in-neighbors”.
Note, however, that the number of edges from [w] to [v] is not necessarily equal
to the number of edges from [v] to [w], as the following example illustrates.
Example. Consider automorphic conjugacy class 6.10. The minimal words
aaaabb, aaabab, and aabaab are vertex representatives for Γ(W ). Neither the au-
tomorphism ({a}, b) nor its inverse are level on any of these three words. Let
φ = ({b}, a). We have φ(aaaabb) ≡ aaabab and φ(aaabab) ≡ aabaab. Note that
φ−1 is not level on aaaabb, so [aaaabb] has outdegree 1. On aabaab, φ has the effect
of φ(aabaab) ≡ abaaab ≡ π(aaabab), where π is the permutation which maps a → a
and b → b, so we have an edge πφ from aabaab to aaabab. Therefore, Γ(W ) with
its vertices labeled is
φ φ
, ,
aaaabb l aaabab ol πφ aabaab .
−1 −1
φ φ
We suppress brackets here to emphasize that we have fixed a representative of each

vertex and that the edge labels are acting on these representatives; in other words,
there are no hidden permutations. As will emerge from the proof of Lemma 3.6,
one can think of Γ(W ) as the path
φ φ φ φ
, , , ,
aaaabb l aaabab l aabaab l abaaab l baaaab
φ−1 φ−1 φ−1 φ−1
folded in half to account for π(aaaabb) ≡ baaaab and π(aaabab) ≡ abaaab. The
symmetry in the center word aabaab allows π(aabaab) ≡ aabaab. Only three of the
four edges between aabaab and its neighbors survive the folding, since π is applied
before φ−1 in φ−1 π(aaabab) ≡ aabaab, so this automorphism does not contribute
an edge to Γ(W ).
It is also possible for a vertex to have a single loop due to a symmetry in a
word.
Example. If w = aababaabb then the automorphism ({b}, a) maps w to the
word ({b}, a)(w) = aabbaabab. Let π map a → a, b → b; since π({b}, a)(w) ≡ w,
the vertex [w] has a loop. However, there is only one loop on [w], since the other
three principal one-letter automorphisms are not level on w. This is class 9.43.
The following are our main theorems. Theorem 2.1 is proved in Section 3, and
Section 4 contains the proofs of Theorems 2.2 and 2.3.
Theorem 2.1. Let W be a non-root class. Then Γ(W ) has one of the following
forms.
(P1) a simple path
( ( ( (
•h •h • ··· • h •h •
possibly in its degenerate form
•
(P2) a looped path

( ( ( (
•h •h • ··· • h •h •d
•d
(P3) a double-edged path
( ( ( (
•h •h • ··· • h • ho •
•Q q
We have referred to the double-looped vertex as a degenerate double-edged

path. This is merely for purposes of convenience; it is not the case that the proof of
Theorem 2.1 will illustrate a sense in which they are related. Alternatively, we could
have given the double-looped vertex its own label and required that double-edged
paths have at least two vertices. However, then we would also have separated the
unlooped vertex and the single-looped vertex from their families, since our proofs
in Section 3 treat them separately as well.
Theorem 2.2. Let W be a root class with no alternating minimal word. Then
Γ(W ) is one of the following graphs.
(R1)
•Q q
(R2)
(/
•h •d
(R3)
6 •V
• jv * •
Theorem 2.3. Let W be a root class containing an alternating minimal word.

Then there is exactly one distinct alternating minimal word modulo J in W ; denote
this word by w0 . Then Γ(W ) is one of the following graphs.
(R4)
7 [w0 ] w
S
(R5)
• gl -* [w ] w
S0
(R6)
•K X aB
BB
BB
BB
B
8 [w0 ]
| |
||
|||
}|
•v
(R7)
•K X 6 •K
x
8 [w0Y ]

•v •
Moreover, each of the ten graph types in Theorems 2.1–2.3 occurs. See Appen-
dix A for examples. Appendix B lists the number of automorphic conjugacy classes
of each graph type for minimal words of length n ≤ 20. Since types (P1)–(P3)
come in different sizes, Appendix C lists the number of paths of each size. Root
classes W , on the other hand, have bounded size |V (Γ(W ))| ∈ {1, 2, 3, 5}.
From this classification it follows that, with the exception of the double-looped
vertex, one can infer from Γ(W ) whether W is a root class or a non-root class.
Furthermore, if W is a root class then one can infer from Γ(W ) whether W contains
an alternating minimal word or not.
Before embarking on the proofs, we mention a distinguished root word.
Example. Let w0 = (abab)n . The image of w0 under ({a}, b) is
({a}, b)(w0 ) = ((ab)b(ba)b)n = (abab)n = w0 .
The other three principal automorphisms map w0 either to (abab)n or (abab)n , so
Γ(W ) is (R4). In fact every class of type (R4) contains (abab)n for some n ≥ 0, so
there is only one such class for each multiple of 4. This can be seen as follows. If w0
is an alternating minimal word of length 4n whose class W has size |V (Γ(W ))| = 1,
then for each one-letter automorphism φ = ({y}, x) the word φ(w0 ) lies in [w0 ]
and is therefore alternating. By Lemma 1.4 we have 0 = (yy)φ(w0 ) = (yxy)w0 ,
which means that no letter y occurs two letters away from itself. It follows that
w0 ≡ σ((abab)n ) for some permutation σ.
The following lemma is key to the proofs of Theorems 2.1–2.3. Under the
condition that w is level under a one-letter automorphism, it provides conditions
for w to be level under the other principal one-letter automorphisms.
Lemma 2.4. Suppose w ∈ C2 such that ({y}, x) is level on w. Then
(i) ({y}, x) is level on w if and only if (yy)w = 0,
(ii) ({x}, y) is level on w if and only if w is a root word, and
(iii) ({x}, y) is level on w if and only if w is an alternating root word.
Proof. Since ({y}, x) is level on w, we have
(2.1) (yx)w = (yx)w + (yy)w
by Lemma 1.5. We use this equation frequently in the following.
By Lemma 1.5, ({y}, x) being level on w is equivalent to (yx)w = (yx)w +(yy)w .
Adding this equation to Equation (2.1) shows that it is equivalent to (yy)w = 0.
This proves (i).
By Lemma 1.5, ({x}, y) being level on w is equivalent to (xy)w = (xy)w +(xx)w ,
which is equivalent to (yx)w = (yx)w +(xx)w . Subtracting this from Equation (2.1)
shows that it is equivalent to (xx)w = (yy)w , which is equivalent to w being a root
word since we also have (yx)w −(yx)w = (yy)w from Equation (2.1). This proves (ii).
Again by Lemma 1.5, ({x}, y) being level on w is equivalent to (xy)w = (xy)w +

(xx)w , which is equivalent to (yx)w = (yx)w +(xx)w . Adding this to Equation (2.1)
shows that it is equivalent to 0 = (xx)w +(yy)w , which is equivalent to 0 = (xx)w =
(yy)w = (yx)w − (yx)w , which is equivalent to w being an alternating root word,
giving (iii).
Lemma 2.4 already provides enough information to restrict the outdegrees of

root word vertices and non-root word vertices.
Corollary 2.5. If w ∈ C2 is a minimal word that is not a root word, then

outdegree([w]) ∈ {0, 1, 2}. If w ∈ C2 is a root word, then outdegree([w]) ∈ {2, 4}.
Proof. We have already established that by definition of Γ(W ) the outdegree

of [w] is at most 4. Suppose toward a contradiction that the outdegree of [w] is 3.
Let ({y}, x) be an automorphism that is level on w. Since the outdegree of each
alternating root word is 4, w is not an alternating root word. By Lemma 2.4, the
automorphism ({x}, y) is therefore not level on w, so the other two automorphisms
({y}, x) and ({x}, y) are level on w. By Lemma 2.4, (yy)w = 0 and w is a root
word. Therefore (xx)w = 0, but this implies that w is alternating and hence an
alternating root word, which is a contradiction. Hence the outdegree of [w] is not
3.
By Lemma 2.4, if w is not a root word then additionally the outdegree is not
4, and if w is a root word then additionally the outdegree is not 1.
It remains to show that if w is a root word then the outdegree of [w] is at least
1. By definition, w is a root word if and only if |(ab)w − (ab)w | = (aa)w = (bb)w ,
in which case (ab)w − (ab)w = (aa)w (and ({a}, b) is level on w by Lemma 1.5) or
(ab)w − (ab)w = (aa)w (and ({a}, b) is level on w).
3. Non-root classes
In this section we prove Theorem 2.1 and Theorem 1.1. For the duration of this
section, fix x, y ∈ L2 such that y ∈
/ {x, x}. We say that a word w is semi-alternating
if (yy)w = 0. We split the proof of Theorem 2.1 into two cases depending on whether
the automorphic conjugacy class contains a semi-alternating minimal word.
Lemma 3.1. Let W be a non-root class that contains no semi-alternating min-

imal word. Then Γ(W ) is one of the following graphs.
• a (P1) path on two vertices,
• a degenerate (P1) path (a single vertex with no edges),
• a degenerate (P2) path (a single vertex with one loop).
Proof. By Lemma 2.4, every vertex in Γ(W ) has outdegree at most 1. On

the other hand, if there is an edge [w] → [v] then there is an edge [v] → [w]. Since
Γ(W ) is connected, it follows that Γ(W ) contains at most 2 vertices. If there are
2 vertices, then Γ(W ) is a simple path on 2 vertices. If there is a single vertex, it
can have either one loop or no loops.
Each of the three possible outcomes in Lemma 3.1 occurs. One can find exam-
ples among words of length 9.
For w ∈ C2 , define mx (w) = min{i ≥ 0 : (yxi y)w ≥ 1}. Similarly, define
mx (w) = min{i ≥ 0 : (yxi y)w ≥ 1}. We adopt the usual convention that min ∅ =
∞. Therefore if (yxi y)w = 0 for all i ≥ 0 then mx (w) = ∞, for example. The
quantity mx (w) is a measure of the “semi-alternatingness” of w. If mx (w) = 0 then
w is not semi-alternating. If mx (w) ≥ 1 then w is semi-alternating and remains so
under mx (w) − 1 applications of ({y}, x).
Lemma 3.2. If w is a minimal word, then 1 ≤ mx (w) < ∞ if and only if
1 ≤ mx (w) < ∞.
Proof. Consider the one-letter automorphism φ = ({y}, x), which maps y →
yx. This automorphism does not change the distance between y and y separated
by xi or xi , since for all i ≥ 0,
φ(yxi y) = yxi y,
φ(yxi y) = xyxi yx,
and analogously for the inverses of these two words. On the other hand, φ does
change the distance between a pair of ys or a pair of ys separated by xi or xi , since
for all i ∈ Z
(3.1) φ(yxi y) = yxi+1 yx
(and analogously for the inverse yx−i y).
Suppose 1 ≤ mx (w) < ∞. Since w is minimal, the image of w under φ has
length at least |w|. Since φ decreases the distance between the two ys in yxmx (w) y
(or the two ys in yxmx (w) y) in w, it follows that φ increases the distance between
another pair of ys or ys in w. This can only happen for yxj y or its inverse for some
j ≥ 0, and since (yy)w = 0 we have 1 ≤ mx (w) < ∞.
A symmetric argument with the automorphism ({y}, x) shows that if 1 ≤
mx (w) < ∞ then 1 ≤ mx (w) < ∞.
Since mx (w) = 0 if and only if mx (w) = 0, it follows from Lemma 3.2 that
mx (w) = ∞ if and only if mx (w) = ∞.
Having proven Lemma 3.1, it remains to prove Theorem 2.1 for classes con-
taining a semi-alternating minimal word. Lemmas 3.4 and 3.6 address the cases
mx (w) = ∞ and 1 ≤ mx (w) < ∞ for the semi-alternating word w. The following
lemma shows that a vertex containing a semi-alternating word has outdegree at
least 2.
Lemma 3.3. Let w be a semi-alternating minimal word of length |w| ≥ 2. Then
({y}, x) and ({y}, x) are level on w.
Proof. Toward a contradiction, assume that neither ({y}, x) nor ({y}, x) is
level on w. If φ = ({y}, x) increases the length of w, then φ causes more additions
than cancellations in w; as in Lemma 1.5, this implies (yx)w < (yx)w + (yy)w .
Symmetrically, |({y}, x)(w)| > |w| implies (yx)w < (yx)w + (yy)w . It follows that
−(yy)w < (yx)w − (yx)w < (yy)w , so (yy)w = 0, contradicting the assumption that
w is semi-alternating. Therefore ({y}, x) or ({y}, x) is level on w. By Lemma 2.4,
both are.
Lemma 3.4. Let W be a non-root class containing a minimal word w such that
mx (w) = ∞. Then Γ(W ) is a degenerate (P3) path (a single vertex with two loops).
Proof. By Lemma 3.3, φ = ({y}, x) and φ−1 = ({y}, x) are level on w. By
Lemma 2.4, φ and φ−1 are the only one-letter automorphisms that are level on w.
Since mx (w) = ∞ and mx (w) = ∞, w consists of overlapping subwords of the form
y e xi y −e for e ∈ {1, −1} and i ∈ Z \ {0}. Since the distance between y e and y −e is
fixed by φ and by φ−1 , w is fixed by φ and by φ−1 , so [w] has two loops.
Suppose that ({x}, y) is level on w. By Lemma 1.5, (xx)w = (xy)w − (xy)w =
(xy)w − (yx)w . This difference is equal to 0 since mx (w) = ∞ implies that the
subwords xy and yx occur in pairs in w and similarly the subwords yx and xy
occur in pairs. But (xx)w = 0 implies that w is an alternating minimal word and
hence a root word by Theorem 1.9, contradicting one of our assumptions. Therefore
({x}, y) is not level on w. Similarly, ({x}, y) = ({x}, y)−1 is not level on w.
We use the following result in the proof of Lemma 3.6.

Lemma 3.5. Let w be a minimal word such that 1 ≤ mx (w) < ∞ and φ =
({y}, x) is level on w. Then φ is level on φ(w) if and only if 2 ≤ mx (w) < ∞.
Proof. Since φ−1 is level on φ(w), we see by Lemma 2.4 that φ is level on
φ(w) if and only if (yy)φ(w) = 0. By Lemma 1.4, (yy)φ(w) = (yxy)w . Since
1 ≤ mx (w) < ∞ by assumption, (yxy)w = 0 if and only if 2 ≤ mx (w) < ∞.
Lemma 3.6. Let W be a non-root class containing a minimal word w such that
1 ≤ mx (w) < ∞. Then Γ(W ) is a (P1), (P2), or (P3) path with at least 2 vertices.
Proof. Lemma 3.3 and Lemma 2.4 imply that φ = ({y}, x) and its inverse
are the only one-letter automorphisms that are level on w. Recall that J is the
subgroup of Aut F2 generated by inner automorphisms and permutations. Let
W = {φj (w) : −mx (w) ≤ j ≤ mx (w)}.
Claim: W ⊂ W , and for each minimal v ∈ W the set W contains a minimal word
equivalent to v modulo J. Note that in W we may have pairs of words that are
equivalent modulo J.
Toward this claim, we first show that for −mx (w) ≤ j ≤ mx (w) the word
φj (w) is minimal, and for −mx (w) < j < mx (w) we also show that φj (w) is
semi-alternating. We work by induction on j. For j = 0, we have by hypothesis
that w is minimal and semi-alternating. Now, suppose that φj (w) is minimal and
semi-alternating for some 0 ≤ j < mx (w). Then φ−1 is level on φj (w), so since
φj (w) is semi-alternating we have that φ is level on φj (w) by Lemma 2.4. Thus,
φj+1 (w) is minimal. It remains to show that if j + 1 < mx (w) then φj+1 (w) is semi-
alternating. In this case, by Equation (3.1) we have mx (φj (w)) = mx (w) − j ≥ 2,
so Lemma 3.5 yields that φj+1 (w) is semi-alternating. A symmetric argument with
φ−1 establishes the cases −mx (w) ≤ j ≤ 0.
In fact φ−mx (w) (w) and φmx (w) (w) are not semi-alternating, since by Equa-
tion (3.1) mx (φmx (w) (w)) = mx (w) − mx (w) = 0. Similarly, mx ((φ−1 )mx (w) (w)) =
0. This means that φ−mx (w) (w) and φmx (w) (w) each have at most one level one-
letter automorphism (again by Lemma 2.4), and in fact φ and φ−1 respectively are
level on these words.
For each −mx (w) ≤ j ≤ mx (w) we have determined the images of φj (w)
under all level automorphisms. Since V (Γ(W )) is connected by level one-letter
automorphisms, W projects onto V (Γ(W )) and the claim follows.
In order to determine Γ(W ) from W , we need to consider the possibility that
some words have been listed in W more than once up to equivalence under ∼J .
For the two endpoint words φ−mx (w) (w) and φmx (w) (w) we have
mx (φ−mx (w) (w)) = mx (w) + mx (w) ≥ 2,
mx (φmx (w) (w)) = mx (w) + mx (w) ≥ 2.
It follows that for u ∈ {φ−mx (w) (w), φmx (w) (w)} we have (xx)u ≥ 1. Since u is also
not semi-alternating, u is not the image of φj (w) under an element of J. Therefore,
at least one minimal word in W is semi-alternating, and at least one but at most
two distinct minimal words modulo J in W are not semi-alternating. So Γ(W ) is a
connected directed graph with either one or two vertices having outdegree 1 and all
other vertices having outdegree 2. Since an edge from [vi ] to [vj ] in Γ(W ) implies
an edge from [vj ] to [vi ], Γ(W ) is one of the paths claimed.
We have completed the proof of Theorem 2.1. The following examples illustrate
the path (P1) of Lemma 3.6.
Example. Class 9.81 contains the word w = aabababab, which for y = b is
semi-alternating. We have ma (w) = 1 and ma (w) = 1, so Γ(W ) for this class is
φ φ
- -
aaabaabbb m aabababab m abbaabaab
−1 −1
φ φ
where φ = ({b}, a). Observe that φ shrinks subwords bai b (and their inverses),
extends subwords bai b (and their inverses), and leaves subwords ba±i b (and their
inverses) fixed. Vertices with outdegree 1 have (bb)w ≥ 1. In each subword bb of
aaabaabbb the automorphism φ introduces a. After applying φ twice, the subword
baab becomes bb, so further applications of φ produce words that are not minimal.
Example. If we begin with a minimal word with (bb)w = 1 rather than (bb)w =
2, then the automorphic conjugacy class can be larger since the word grows at only
one position rather than two. For example, consider the word aaabababb belonging
to class 9.97. Its graph Γ(W ) is
φ φ φ
- - -
aaabababb m aabababab m abababaab m bababaaab
φ−1 φ−1 φ−1
where again φ = ({b}, a).

The automorphic conjugacy classes that are most relevant for Theorem 1.1 are
those addressed by Theorem 2.1. Therefore we now give a proof of Theorem 1.1,
even though Theorems 2.2 and 2.3 on which it depends will be proved in Section 4.
Proof of Theorem 1.1. If W is a root class whose minimal words have

length n ≥ 9, then in fact n ≥ 12 by Theorem 1.7; by Theorems 2.2 and 2.3,
|V (Γ(W ))| ≤ 5 ≤ n − 5.
Therefore let W be a non-root class whose minimal words have length n ≥ 9.

We may assume that W contains a minimal word v with 1 ≤ mx (v) < ∞, since
otherwise |V (Γ(W ))| ≤ 2 by Lemmas 3.1 and 3.4. By the proof of Lemma 3.6,
|V (Γ(W ))| ≤ mx (v) + 1 + mx (v)

≤ 1 + max{i : xi appears in a minimal word in W }.
Therefore it suffices to show that if xi appears in a minimal word w of length n

and n − 6 < i ≤ n − 1 then |V (Γ(W ))| ≤ n − 5.
By applying a permutation, we may assume x = a and y = b, so w ≡ an−1 b
or w ≡ an−k bub for some subword u of length k − 2 ≤ 3. The word an−1 b is not
minimal, so it suffices to consider an−k bub. There are sufficiently few possibilities
for u that we simply check them all.
If u is the empty word, then w ≡ an−2 bb. This word is minimal, and its graph
Γ(W ) is of type (P2) for odd n and of type (P3) for even n. The number of vertices
in Γ(W ) is n/2, which satisfies n/2 ≤ n − 5 for n ≥ 9,
There are 3 words of length 1 to check. If u = a then w is not minimal. If
u = a then w ∼ an−2 bb so we have already shown that the graph has at most n − 5
vertices. If u = b then w is minimal, and Γ(W ) is (P1) of size 1.
There are 7 words of length 2 to check:
u Γ(W )
aa not minimal
ab not minimal
ba not minimal
bb (P1) of size 1
ba (P1) of size 1
ab (P1) of size 1
aa (P2) or (P3) of size n/2
Finally, there are 21 words of length 3 to check:
u Γ(W ) u Γ(W )
aaa not minimal bba (P1) of size 1
aab not minimal bab (P1) of size 1
aba not minimal baa (P1) of size 1
abb (P1) of size 2 aba not minimal
aba not minimal abb (P1) of size 1
aba not minimal aba not minimal
aba not minimal aab (P1) of size 1
baa not minimal aaa (P2) or (P3) of size n/2
bab (P1) of size 2 aba not minimal
bba (P1) of size 2 aba not minimal
bbb (P1) of size 1
Hence |V (Γ(W ))| ≤ n − 5 for all minimal words an−k bub of length n where
2 ≤ k ≤ 5, and the statement follows.
Theorem 1.1 is sharp in the sense that for every n ≥ 9 there exists an automor-
phic conjugacy class W with minimal words of length n such that |V (Γ(W ))| = n−5.
For example, the class containing an−6 bababb is a (P1) class with distinct vertex
representatives an−6−j bababaj b for 0 ≤ j ≤ n − 6. There appear to be 5 such (P1)
classes for each n ≥ 9; see Section 5 and Appendix C.
As can be observed from the data in Appendix C, when n is odd the double-
edged path occurs only in its degenerate form.
Corollary 3.7. Let W be a non-root class containing a minimal word of odd
length such that Γ(W ) is of type (P3). Then |V (Γ(W ))| = 1.
Proof. Let [w] be the endpoint with outdegree 2 of a nondegenerate (P3)
graph. By the proof of Lemma 3.6, φ(w) ∼J φ−1 (w) for some one-letter automor-
phism φ = ({y}, x) that is level on w. Write πφ(w) ≡ φ−1 (w) for some permutation
π. Since w is semi-alternating and (xx)w = 0, π(x) ∈ {x, x} and π(y) ∈ {y, y}.
If π maps x → x, y → y or x → x, y → y, then by Lemma 1.2 φπ(w) ≡ φ−1 (w),
so π(w) ≡ φ−2 (w), which contradicts [w] being an endpoint. Therefore π maps
x → x, y → y or x → x, y → y. By Lemma 1.2, φ−1 π(w) ≡ φ−1 (w), so w has
a symmetry π(w) ≡ w. Let k ≥ 1 be minimal such that w = ρk π(w), where
ρ is rotation to the right by one character. Let u be the prefix of w of length
k. Then w = u · π(u) · π 2 (u) · π 3 (u) · · · π −1 (u). Since π has order 2, we have
w = (u · π(u))|w|/(2k) and |w| is even.
4. Root classes
In this section we prove Theorems 2.2 and 2.3, establishing the structure of root
classes. For this, we need a lemma concerning the composition of two one-letter
automorphisms. Note that we compose functions from right to left, as in Section 1.
Lemma 4.1. Let x, y ∈ L2 with y ∈
/ {x, x}. Let π be the permutation which
maps x → y and y → x. Then
({x}, y) · ({y}, x) = π · ({x, x}, y) · ({x}, y).
Proof. One checks that both sides map x → yx and y → x.
A consequence of Lemma 4.1 is that [({x}, y)({y}, x)(w)] = [({x}, y)(w)] for all
w ∈ C2 . That is, the vertex [({x}, y)({y}, x)(w)] is a neighbor of [w] in Γ(W ).
Now we determine the structure of root classes with no alternating word.
Proof of Theorem 2.2. Let W be a root class with no alternating minimal
word.
By Corollary 2.5, the outdegree of a root word vertex [w] is either 2 or 4.
If w is not alternating, then by Lemma 2.4 there are only two level one-letter
automorphisms on w. Therefore every vertex in Γ(W ) has outdegree 2.
We show that any two distinct vertices in Γ(W ) are neighbors. Suppose that
u, v, w ∈ W are minimal words such that v ≡ φ(w) and u ≡ ψ(v) ≡ ψφ(w) for some
one-letter automorphisms φ = ({y}, x) and ψ. We want to show that either [w] = [u]
or [w] is connected to [u] by a one-letter automorphism. This will then imply that
any two vertices that are connected by a sequence of one-letter automorphisms are
either the same vertex or are in fact connected by a single one-letter automorphism.
We know that φ−1 = ({y}, x) is level on v. Since v is a root word which is
not alternating, we have (xx)v = (yy)v = 0 and therefore Lemma 2.4 implies that
φ−1 and ({x}, y) are the only (distinct modulo Inn F2 ) one-letter automorphisms
that are level on v. Since ψ is level on v, ψ is equivalent modulo Inn F2 to either
φ−1 or ({x}, y). There are therefore two cases. If ψ is equivalent to φ−1 , then we
have w ≡ u. If instead ψ is equivalent to ψ = ({x}, y), then by Lemma 4.1 we
have ψφ(w) ≡ ψ φ(w) = π({x, x}, y)({x}, y)(w), where π is the permutation which
maps x → y and y → x; this implies that [w] is connected to [ψφ(w)] = [u] by a
one-letter automorphism.
We have shown that if w and u are minimal words in W , then [w] = [u] or
[w] and [u] are neighbors. Since the outdegree of each vertex in Γ(W ) is 2, this
implies that there are at most three vertices in Γ(W ). If |V (Γ(W ))| = 1, then
Γ(W ) is (R1), a single vertex with two loops. If |V (Γ(W ))| = 3, then Γ(W ) is (R3),
a bi-directed 3-cycle. Otherwise, |V (Γ(W ))| = 2. Let [w] and [φ(w)] be the two
vertices of Γ(W ). There is a directed edge from [w] to [φ(w)] and another from
[φ(w)] to [w], so it suffices to determine the other two edges. As above, ({x}, y)
is level on φ(w) and not equivalent modulo Inn F2 to φ−1 , so this automorphism
contributes an edge from [φ(w)] to [({x}, y)(w)], which is one of the two vertices.
By Lemma 4.1, there is a directed edge from [w] to [({x}, y)(w)]. Therefore the
other two edges point to the same vertex, and Γ(W ) is (R2).
Example. Let W be class 8.37, whose graph is (R3). Let π be the permutation
mapping a → b, b → a. Write φyx = ({y}, x). Then Γ(W ) is the following graph,
where an edge w → v labeled φ satisfies φ(w) ≡ v.
ababaabb
5 `
φab πφba
π −1 φab
v
φab
φba - !
aaababbb m aabbabab
φba
Now we start with alternating words. We need several lemmas.
Lemma 4.2. Suppose w0 is an alternating minimal word and φ is a one-letter

automorphism such that φ(w0 ) is an alternating minimal word. Then φ(w0 ) = w0 .
Proof. Write φ = ({y}, x). Since φ(w0 ) is alternating, we have (yy)φ(w0 ) = 0,

so (yxy)w0 = 0 by Lemma 1.4. The only length-2 subwords that cause cancellations
under φ are yx and xy. Since (yxy)w0 = 0 and w0 is alternating, every yx in w0
appears in yxy and every xy appears in yxy. But φ(yxy) = yxy and φ(yxy) = yxy,
so φ causes no cancellations in w0 . Since all one-letter automorphisms are level on
w0 by Theorem 1.9, φ also causes no additions in w0 . Therefore φ(w0 ) = w0 .
For the rest of this section, denote φ1 = ({y}, x), φ2 = φ−1 1 = ({y}, x),
φ3 = ({x}, y), and φ4 = φ−1 3 = ({x}, y). These are four principal one-letter au-
tomorphisms, and they are distinct modulo Inn F2 . In this notation, Lemma 4.1
implies that [φ4 φ1 (w)] = [φ3 (w)]. We record this in the following corollary, along
with analogous statements obtained by applying permutations to L2 .
Corollary 4.3. For w ∈ C2 ,

[φ2 φ3 (w)] = [φ1 (w)],
[φ1 φ4 (w)] = [φ2 (w)],
[φ4 φ1 (w)] = [φ3 (w)],
[φ3 φ2 (w)] = [φ4 (w)].
The statements of the next three lemmas are all of the same form. They
determine the neighborhood of a vertex containing an alternating minimal word.
They form the bulk of the proof of Theorem 2.3. Recall from Theorem 1.9 that all
one-letter automorphisms are level on alternating minimal words.
Lemma 4.4. Let w0 be an alternating minimal word such that [φ1 (w0 )] =
[φ2 (w0 )] for some x, y ∈ L2 with y ∈
/ {x, x}. Then [φ3 (w0 )] = [φ4 (w0 )].
Proof. By Theorem 1.9, φ1 is level on w0 . Since φ2 = φ−1 1 , φ2 is level on
φ1 (w0 ). By Lemma 2.4, φ4 is also level on φ1 (w0 ). Let us compute the neighbors
of φ1 (w0 ) under φ2 and φ4 . We have [φ2 φ1 (w0 )] = [w0 ], and Corollary 4.3 implies
that [φ4 φ1 (w0 )] = [φ3 (w0 )]. Similarly, the images of φ2 (w0 ) under φ1 and φ3 are
[φ1 φ2 (w0 )] = [w0 ] and [φ3 φ2 (w0 )] = [φ4 (w0 )].
If φ1 (w0 ) is alternating, then φ1 (w0 ) = w0 by Lemma 4.2. Then [φ4 φ1 (w0 )] =
[φ3 (w0 )] implies [φ4 (w0 )] = [φ3 (w0 )] as desired.
If φ1 (w0 ) is not alternating, then by Lemma 2.4 the outdegree of [φ1 (w0 )] is 2.
Since we have shown that [w0 ], [φ3 (w0 )], and [φ4 (w0 )] are all neighbors of [φ1 (w0 )],
it follows that two of these three vertices are equal. If [φ3 (w0 )] = [φ4 (w0 )], we are
finished. If [w0 ] = [φ3 (w0 )] or [w0 ] = [φ4 (w0 )], then we see that φ3 (w0 ) or φ4 (w0 )
is alternating; in either case Lemma 4.2 gives φ3 (w0 ) = w0 = φ4 (w0 ).
/ {x, x}. Then [φ2 (w0 )] = [φ4 (w0 )].
Proof. By the definition of a root word, (yy)φ1 (w0 ) = (xx)φ1 (w0 ) ; rewriting
each side using Lemma 1.4 gives
(yxy)w0 = (yxy)w0 + (yxx)w0 + (xxy)w0 + (xxx)w0 .
Since w0 is alternating, this equation becomes (yxy)w0 = (yxy)w0 . Symmetrically,
since φ3 (w0 ) is a root word, we have (xyx)w0 = (xyx)w0 .
Let π be a permutation such that φ1 (w0 ) ≡ πφ3 (w0 ). Then (xx)φ1 (w0 ) =
(xx)πφ3 (w0 ) = (xx)φ3 (w0 ) , so
(yxy)w0 = (yxy)w0 = (xyx)w0 = (xyx)w0 .
For six of the eight possible permutations π, we show that these four expressions
are equal to 0. For these π, this will imply that no letter occurs two letters away
from itself in w0 , so w0 ≡ σ((abab)n ) for some permutation σ. As already stated in
Section 2, for this word we have [φ(w0 )] = [w0 ] for each one-letter automorphism
φ.
If π maps x → x, y → y or x → x, y → y, consider (yxy)φ1 (w0 ) = (yxy)πφ3 (w0 ) .
Then (yxy)φ1 (w0 ) = (yxy)φ3 (w0 ) , and rewriting each side gives
(yy)w0 = (yxy)w0 + (yxx)w0 + (xxy)w0 + (xxx)w0 ,
which simplifies to 0 = (yxy)w0 because w0 is alternating.
If π maps x → x, y → y or x → x, y → y, then consider (yxy)φ1 (w0 ) =

(yxy)πφ3 (w0 ) . Since (yxy)πφ3 (w0 ) = (yxy)φ3 (w0 ) , the right side is the same as before,
and we obtain
(yxxy)w0 = (yxy)w0 + (yxx)w0 + (xxy)w0 + (xxx)w0 ,
which simplifies to 0 = (yxy)w0 .
If π maps x → y, y → x or x → y, y → x, use Lemma 1.2 to write φ1 (w0 ) ≡
πφ3 (w0 ) = ({π(x)}, π(y))π(w0). In either case, we obtain φ1 (w0 ) ≡ φ2 π(w0 ) (where
for the permutation x → y, y → x we have used Equation (1.1)). Hence φ21 (w0 ) ≡
π(w0 ), and φ21 (w0 ) is alternating. In particular, (yxxxy)φ21 (w0 ) = 0, and this implies
(yxy)w0 = 0.
Two permutations remain to be considered. Let π map x → y, y → x or
x → y, y → x. Lemma 1.2 gives φ1 (w0 ) ≡ πφ3 (w0 ) = ({π(x)}, π(y))π(w0) ≡
φ1 π(w0 ). Hence w0 ≡ π(w0 ). We show that the only alternating minimal word
satisfying this equation is the empty word. Assume toward a contradiction that w0
is nonempty. Let k ≥ 1 be minimal such that w0 = ρk π(w0 ), where ρ is rotation
to the right by one character. Let u be the prefix of w0 of length k. Then w0 =
u·π(u)·π 2 (u)·π 3 (u) · · · π −1 (u). Since π has order 2, we have w0 = (u·π(u))|w0 |/(2k)
and |w0 | is divisible by 2k. Since w0 is alternating and π(x) ∈ {y, y}, k is odd.
Since w0 is a root word, it follows that u · π(u) is a root word. By Theorem 1.7,
|u · π(u)| = 2k is divisible by 4, which contradicts k being odd.
As we have just seen, (abab)n is essentially the only alternating minimal word
satisfying [φ1 (w0 )] = [φ3 (w0 )]. However, the equation [φ1 (w0 )] = [φ4 (w0 )], which
is the subject of the following lemma, has additional solutions. For example,
abababababab is a solution.
/ {x, x}. Then [φ2 (w0 )] = [φ3 (w0 )].
Proof. As in the proof of Lemma 4.5, one can show that
(yxy)w0 = (yxy)w0 = (xyx)w0 = (xyx)w0 .
Write φ1 (w0 ) ≡ πφ4 (w0 ). For six of the eight possible permutations π, we now
show that these four expressions are equal to 0; it will follow in these cases that
w0 ≡ σ((abab)n ) for some permutation σ, and hence [φ2 (w0 )] = [φ3 (w0 )].
If π maps x → x, y → y or x → x, y → y, consider (yxy)φ1 (w0 ) = (yxy)πφ4 (w0 ) .
This is equivalent to
(yxxy)w0 = (yxy)w0 + (yxx)w0 + (xxy)w0 + (xxx)w0 ,
which simplifies to 0 = (yxy)w0 since w0 is alternating.
If π maps x → x, y → y or x → x, y → y, consider (yxy)φ1 (w0 ) = (yxy)πφ4 (w0 ) =
(yxy)φ4 (w0 ) . Therefore 0 = (yxy)w0 .
If π maps x → y, y → x or x → y, y → x, then by Lemma 1.2 we have φ1 (w0 ) ≡
πφ4 (w0 ) = ({π(x)}, π(y))π(w0 ) ≡ φ2 π(w0 ). As in the proof of Lemma 4.5, φ21 (w0 ) ≡
π(w0 ) implies (yxy)w0 = 0.
It remains to address the two order-4 permutations mapping x → y, y → x and
x → y, y → x. Let π be either of these permutations. By Lemma 1.2, φ1 (w0 ) ≡
πφ4 (w0 ) ≡ φ1 π(w0 ). Hence w0 ≡ π(w0 ). Since the conclusion holds for the empty
word, assume w0 is nonempty. Let k ≥ 1 be minimal such that w0 = ρk σ(w0 ) for
some σ ∈ {π, π −1 }, where again ρ is rotation to the right by one character. Let u
be the prefix of w0 of length k. Then w0 = u · σ(u) · σ 2 (u) · σ 3 (u) · · · σ −1

(u). Since

3 n
σ has order 4, we have w0 = (u · σ(u) · σ (u) · σ (u)) =
2 3 n
i=0 σ (u)
i
, where
|w0 |
n= 4k . Therefore σ(w0 ) ≡ w0 . By Lemma 1.2 and Equation (1.1),
σφ2 (w0 ) = ({σ(y)}, σ(x))σ(w0 )

≡ φ3 σ(w0 )
≡ φ3 (w0 ),
so [φ2 (w0 )] = [φ3 (w0 )].
Experimental evidence suggests that in fact the previous three lemmas can be
generalized, but we do not have a proof.
Conjecture. Lemmas 4.4, 4.5, and 4.6 remain true if we remove the require-
ment that w0 is alternating.
For example, aaaa satisfies the condition [φ1 (w0 )] = [φ2 (w0 )] of Lemma 4.4
and also the conclusion [φ3 (w0 )] = [φ4 (w0 )]. Examples for Lemmas 4.5 and 4.6 are,
respectively, aabb and aabbaabb.
To classify the graphs of root classes containing an alternating minimal word,
however, we only need the lemmas as stated.
Proof of Theorem 2.3. First we establish the uniqueness of an alternating

word vertex [w0 ] in Γ(W ) and that every other vertex is a neighbor of [w0 ]. Let
w0 ∈ W be an alternating root word. If [w0 ] is the only vertex of Γ(W ), then it
is clearly the unique vertex containing alternating minimal words. Otherwise, let
x, y ∈ L2 such that [φ1 (w0 )] = [w0 ]. By Lemma 4.2, φ1 (w0 ) is not alternating.
Thus, by Lemma 2.4, the outdegree of [φ1 (w0 )] is 2. As in the proof of Lemma 4.4,
the principal automorphisms that are level on φ1 (w0 ) are φ2 and φ4 . The image of
φ1 (w0 ) under φ2 is w0 , and by Corollary 4.3 the vertex [φ4 φ1 (w0 )] is connected to
[w0 ] by a one-letter automorphism. That is, any edge from [φ1 (w0 )] that does not
point to [w0 ] points to a neighbor of [w0 ] (possibly to [φ1 (w0 )] itself). Since Γ(W )
is connected, this implies that every vertex other than [w0 ] is, in fact, a neighbor
of [w0 ]. Lemma 4.2 now implies that [w0 ] is the unique vertex in Γ(W ) containing
an alternating minimal word.
By Lemma 4.2, if [w0 ] is connected to itself by [φ] for some one-letter automor-
phism φ then [w0 ] is also connected to itself by [φ−1 ]. In other words, loops on [w0 ]
come in pairs of inverse automorphisms. We consider separately the cases that [w0 ]
has 4, 2, or 0 loops.
If [w0 ] has 4 loops, then Γ(W ) is (R4), a single vertex with four loops.
Suppose [w0 ] has exactly 2 loops. Let x, y be such that [φ1 (w0 )] = [w0 ] =
[φ2 (w0 )]. By Lemma 4.4, [φ3 (w0 )] = [φ4 (w0 )], so Γ(W ) has exactly two vertices,
[w0 ] and [φ3 (w0 )]. Since φ3 and φ4 are inequivalent modulo Inn F2 , two edges
connect [w0 ] to [φ3 (w0 )]. This accounts for all four edges emanating from [w0 ], so
it suffices to determine the edges from [φ3 (w0 )]. The one-letter automorphisms that
are level on φ3 (w0 ) are φ4 and φ2 . Moreover, φ4 φ3 (w0 ) = w0 and by Corollary 4.3
[φ2 φ3 (w0 )] = [φ1 (w0 )] = [w0 ]. There are therefore two edges from [φ3 (w0 )] to [w0 ],
so Γ(W ) is (R5).
Finally, suppose that [w0 ] has no loops. Corollary 4.3 implies that [φ1 (w0 )] is
connected to [φ3 (w0 )] by a one-letter automorphism and that [φ2 (w0 )] is connected
to [φ4 (w0 )] by a one-letter automorphism (allowing the possibility that these edges
may be loops). If [w0 ] has four distinct neighbors, then, since [w0 ] is the only vertex
with outdegree 4, the outdegree of each other vertex is 2, and it follows that Γ(W )
is the bow tie (R7). If [w0 ] has fewer than four neighbors, then there is at least one
pair of identified images of w0 . The 42 = 6 possibilities are as follows.
If [φ1 (w0 )] = [φ2 (w0 )], then [φ3 (w0 )] = [φ4 (w0 )] by Lemma 4.4. Therefore [w0 ]
has exactly two neighbors, each of which has outdegree 2. Moreover, two edges
connect [w0 ] to each of its neighbors. Therefore Γ(W ) is (R6).
If [φ1 (w0 )] = [φ3 (w0 )], then the proof of Lemma 4.5 shows that φ1 (w0 ) is
alternating. Therefore φ1 (w0 ) = w0 by Lemma 4.2, contradicting our assumption
that w0 has no loops.
If [φ1 (w0 )] = [φ4 (w0 )], then [φ2 (w0 )] = [φ3 (w0 )] by Lemma 4.6. The vertices
of Γ(W ) are as in the case [φ1 (w0 )] = [φ2 (w0 )], with analogous edges, so Γ(W ) is
(R6).
The remaining three cases are equivalent under permutations to the first three.
If [φ3 (w0 )] = [φ4 (w0 )], then let σ be the permutation that maps x → y, y → x.
Then [φ1 σ(w0 )] = [φ2 σ(w0 )], which is the first case we considered, so Γ(W ) is (R6).
If [φ2 (w0 )] = [φ4 (w0 )], letting σ map x → x, y → y gives [φ1 σ(w0 )] = [φ3 σ(w0 )],
which is the second case and so does not occur when [w0 ] has no loops.
If [φ3 (w0 )] = [φ2 (w0 )], then [φ1 σ(w0 )] = [φ4 σ(w0 )], where σ maps x → y, y →
x. This is the third case, so Γ(W ) is (R6).
5. Enumeration
Having classified automorphic conjugacy classes of F2 in this paper, it is natural
to ask how many automorphic conjugacy classes contain minimal words of length
n. In this section we make some observations that suggest the intriguing possibility
of an exact enumeration. We restrict our speculation to non-root classes, which
outnumber root classes (at least for 5 ≤ n ≤ 20 and probably for n > 20 as well).
In Section 3 we mentioned that for 9 ≤ n ≤ 20 there are precisely 5 (P1) classes
of size |V (Γ(W ))| = n − 5 (the largest possible size, per Theorem 1.1). This can
be clearly seen in Appendix C as an eventually constant diagonal of 5s in the table
enumerating (P1) classes. Our first conjecture is that all diagonals of this table are
eventually constant. The tables enumerating (P2) and (P3) classes, which result
from folding, suggest that these classes have size at most n/2 for n ≥ 2, so we
phrase the conjecture as follows.
Conjecture. Fix k ≥ 0. The number of automorphic conjugacy classes of F2

of size n − k whose minimal words have length n is constant for sufficiently large
n.
For k = 0, 1, 2, . . . , these constants appear to be
(5.1) 0, 0, 0, 0, 0, 5, 12, 17, 24, 67, 196, 437, . . . .
A simple expression for the kth term of this sequence is not obvious. However,
refining our parameterization of classes reveals additional structure.
Define the weight of a word w to be min((a)w , (b)w ). Suppose φ = ({y}, x) is

level on a minimal word w of length n. Then (y)w = (y)φ(w) , and hence (x)w =
n − (y)w = n − (y)φ(w) = (x)φ(w) . Therefore the weight of a minimal word is
preserved under level one-letter automorphisms. The weight is also preserved under
inner automorphisms and permutations, so the weight is invariant on all minimal
words in an automorphic conjugacy class W .
Let us count classes not by size alone but by size and weight. There is only one
class of weight 0 for each n ≥ 0, namely the class containing an , which has size 1.
There are no classes of weight 1, since an−1 b is not minimal.
We return to (P1) classes. For 9 ≤ n ≤ 20, the 5 classes of type (P1) and
size n − 5 all have weight 4. Similarly, for 10 ≤ n ≤ 20, all 12 second-largest (P1)
classes (those of size n − 6) have weight 4. The 17 third-largest classes all have
weight 4, and the 24 fourth-largest classes also all have weight 4. However, not all
67 fifth-largest classes have weight 4; it turns out that 29 have weight 4 and 38
have weight 6. If, instead of Sequence (5.1), we consider the number of classes (for
sufficiently large n) of size n − k whose minimal words have length n and weight 4,
we obtain the sequence
0, 0, 0, 0, 0, 5, 12, 17, 24, 29, 36, 41, . . . ,

whose terms are given by a simple expression. Namely, this sequence is eventually
a linear quasi-polynomial with modulus 2.
Conjecture. For k ≥ 4 and n ≥ max(2k − 2, 9), the number of (P1) classes

of size n − k whose minimal words have length n and weight 4 is

6k − 24 if k ≡ 0 mod 2,
6k − 25 if k ≡ 1 mod 2.
It appears that all classes of odd weight have size 1. For even weights, however,
we see behavior similar to weight-4 classes. For example, fixing k, the number of
(P1) classes of size n − k and weight 6 appears to be constant for n ≥ 2k − 5,
with values 38, 160, 396, 800 for k = 9, . . . , 12. These four terms are not enough
to guess a reliable expression for the kth term, but we suspect it is given by a
quasi-polynomial as well.
Therefore it seems that sufficiently large (P1) classes should be amenable to
enumeration. Analogous conjectures for (P2) and (P3) classes aren’t quite as
strongly suggested by the data available in Appendix C, but we are still willing
to state the following.
Conjecture. Fix an odd k ≥ 1. The number of (P2) classes of size (n − k)/2

whose minimal words have length n is constant for sufficiently large odd n.
Conjecture. Fix an even k ≥ 0. The number of (P3) classes of size (n−k)/2

whose minimal words have length n is constant for sufficiently large even n.
On the other side of the spectrum, counting small classes as opposed to large
classes seems promising as well. Let us consider classes of size 1, which for 0 ≤
n ≤ 20 account for more than half of all classes whose minimal words have length
n (nearly 88% for n = 20). For odd weights, the number of size-1 classes appears
to be given by a polynomial.
Conjecture. For n ≥ 7, the number of non-root classes of size 1 whose

minimal words have length n and weight 3 is 3n − 11.
minimal words have length n and weight 5 is
1
35n3 − 645n2 + 3988n − 8262 .
6
For even weights, the expressions seem to be quasi-polynomials rather than
polynomials.

n − 2 if n ≡ 0 mod 2,
n − 3 if n ≡ 1 mod 2.
⎧ 3
⎪
⎪ 2n − 36n2 + 244n − 540 /6 if n≡0 mod 4,
⎨2n3 − 36n2 + 241n − 537 /6
⎪
if n≡1 mod 4,
3
⎪
⎪ 2n − 36n2 + 244n − 546 /6 if n≡2 mod 4,
⎪
⎩
2n3 − 36n2 + 241n − 537 /6 if n≡3 mod 4.
We leave these conjectures and their generalizations as open problems. The
referee has pointed out that, aside from independent interest, knowing the number
of automorphic conjugacy classes of a given size would allow one to compute the
expected size |V (Γ(W ))| of a random class W whose minimal words have length n.
There are sufficiently many classes of size 1 that for each 0 ≤ n ≤ 20 this number
lies in the interval [1, 1.76), with the value for n = 20 being approximately 1.18.
Does the expected size of a random class lie in the interval [1, 2) for all n ≥ 0? Does
the expected size of a random class tend to 1 as n gets large?
Appendix A. Table of automorphic conjugacy classes

The following tables list all automorphic conjugacy classes containing a word
of length n ≤ 9. For a given length, classes are sorted first by size and then by the
lexicographically least word. Representatives modulo J of minimal words in each
class are given, and each class is identified by its graph type in Theorems 2.1, 2.2,
and 2.3. Data files listing all automorphic conjugacy classes containing a word of
length n ≤ 20 can be downloaded from the second author’s web site1 .
0.1 (R4) 7.7 (P1) aaababb 8.20 (P1) aaabbaab

1.1 (R1) a 7.8 (P1) aaababb 8.21 (P1) aaabbaab
2.1 (P3) aa 7.9 (P1) aaabbab 8.22 (P1) aaabbabb
3.1 (P3) aaa 7.10 (P1) aaabbab 8.23 (P1) aaabbabb
4.1 (P3) aaaa 7.11 (P1) aaabbab 8.24 (P1) aaabbabb
4.2 (R4) abab 7.12 (P3) aaabaab 8.25 (P1) aaabbbab
4.3 (R5) aabb 7.13 (P1) aabaabb 8.26 (P1) aaabbbab
abab 7.14 (P1) aabbaab 8.27 (P3) aaabaaab
5.1 (P3) aaaaa 7.15 (P1) aabbabb 8.28 (P1) aabbaabb
5.2 (P3) aabab 7.16 (P2) aaaaabb 8.29 (P1) aabbaabb
5.3 (P3) aabab aaaabab 8.30 (R1) aabbabab
5.4 (P2) aaabb aaabaab 8.31 (R4) abababab
aabab 8.1 (P3) aaaaaaaa 8.32 (R2) aabababb
6.1 (P3) aaaaaa 8.2 (P3) aaaaabab aababbab
6.2 (P3) aaabab 8.3 (P1) aaaaabbb 8.33 (R2) aabababb
6.3 (P1) aaabbb 8.4 (P3) aaaaabab aababbab
6.4 (P3) aaabab 8.5 (P3) aaaabaab 8.34 (R5) aabbaabb
6.5 (P3) aabaab 8.6 (P1) aaaababb abababab
6.6 (P1) aababb 8.7 (P1) aaaababb 8.35 (R5) aabbabab
6.7 (P1) aabbab 8.8 (P1) aaaababb abababab
6.8 (P1) aabbab 8.9 (P1) aaaabbab 8.36 (R5) aababbab
6.9 (P3) aabaab 8.10 (P1) aaaabbab abababab
6.10 (P3) aaaabb 8.11 (P1) aaaabbab 8.37 (R3) aaababbb
aaabab 8.12 (P1) aaaabbbb aabababb
aabaab 8.13 (P3) aaaabaab aabbabab
7.1 (P3) aaaaaaa 8.14 (P3) aaabaaab 8.38 (R6) aaabbabb
7.2 (P3) aaaabab 8.15 (P1) aaabaabb aababbab
7.3 (P1) aaaabbb 8.16 (P1) aaabaabb abababab
7.4 (P3) aaaabab 8.17 (P1) aaabaabb 8.39 (R3) aababbab
7.5 (P3) aaabaab 8.18 (P1) aaababbb aababbab
7.6 (P1) aaababb 8.19 (P1) aaabbaab aabababb
1 http://thales.math.uqam.ca/
~rowland/data/automorphic_conjugacy_classes.html as of
this writing.
8.40 (R3) aababbab 9.17 (P1) aaaabaabb 9.49 (P3) aabababab

aabababb 9.18 (P1) aaaababbb 9.50 (P3) aabababab
aabbabab 9.19 (P1) aaaababbb 9.51 (P3) aabababab
8.41 (P3) aaaaaabb 9.20 (P1) aaaababbb 9.52 (P3) aabababab
aaaaabab 9.21 (P1) aaaabbaab 9.53 (P1) aaaababbb
aaaabaab 9.22 (P1) aaaabbaab aaabbabab
aaabaaab 9.23 (P1) aaaabbaab 9.54 (P1) aaaabbabb
8.42 (R7) aabababb 9.24 (P1) aaaabbabb aaababbab
aabbabab 9.25 (P1) aaaabbabb 9.55 (P1) aaaabbbab
aabbabab 9.26 (P1) aaaabbabb aaabababb
aabbabab 9.27 (P1) aaaabbbab 9.56 (P1) aaabababb
abababab 9.28 (P1) aaaabbbab aaabbabab
8.43 (R7) aabababb 9.29 (P1) aaaabbbab 9.57 (P1) aaababbab
aababbab 9.30 (P3) aaaabaaab aaabbabab
aabbabab 9.31 (P1) aaabaaabb 9.58 (P1) aaababbab
aababbab 9.32 (P1) aaabaabbb aababbabb
abababab 9.33 (P1) aaabbaaab 9.59 (P1) aaababbab
9.1 (P3) aaaaaaaaa 9.34 (P1) aaabbaaab aaabbabab
9.2 (P3) aaaaaabab 9.35 (P1) aaabbaabb 9.60 (P1) aaabababb
9.3 (P1) aaaaaabbb 9.36 (P1) aaabbaabb aababaabb
9.4 (P3) aaaaaabab 9.37 (P1) aaabbaabb 9.61 (P1) aaabababb
9.5 (P3) aaaaabaab 9.38 (P1) aaabbabbb aaabbabab
9.6 (P1) aaaaababb 9.39 (P1) aaabbbaab 9.62 (P1) aaabababb
9.7 (P1) aaaaababb 9.40 (P1) aaabbbaab aaababbab
9.8 (P1) aaaaababb 9.41 (P1) aaabbbabb 9.63 (P1) aaabababb
9.9 (P1) aaaaabbab 9.42 (P1) aaabbbabb aaababbab
9.10 (P1) aaaaabbab 9.43 (P2) aababaabb 9.64 (P2) aaababbab
9.11 (P1) aaaaabbab 9.44 (P3) aabababab aabababab
9.12 (P1) aaaaabbbb 9.45 (P3) aabababab 9.65 (P1) aaabbabab
9.13 (P3) aaaaabaab 9.46 (P3) aabababab aaababbab
9.14 (P3) aaaabaaab 9.47 (P3) aabababab 9.66 (P1) aaabbabab
9.15 (P1) aaaabaabb 9.48 (P2) aabbaabab aabbabbab
9.16 (P1) aaaabaabb
9.67 (P1) aaabbabab 9.82 (P1) aaabbaabb 9.93 (P1) aabbaabab

aaababbab aabbaabab aabbabaab
9.68 (P1) aaabbabab aabababab aabababab
aaabababb 9.83 (P1) aaabbabbb 9.94 (P1) aabbaabab
9.69 (P1) aaabbabab aabaabbab aabbabbab
aabbababb aabababab aabababab
9.70 (P1) aaabbabab 9.84 (P1) aabaababb 9.95 (P1) aabbabaab
aaabababb aabaabbab aabbabbab
9.71 (P1) aaabababb aabababab aabababab
aababbabb 9.85 (P1) aabaabbab 9.96 (P2) aaaaaaabb
9.72 (P1) aaababbab aabababab aaaaaabab
aababbaab aabbababb aaaaabaab
9.73 (P1) aaabababb 9.86 (P1) aabaabbab aaaabaaab
aaababbab aabababab 9.97 (P1) aaabababb
9.74 (P1) aaabababb aababaabb aaabbabab
aaababbab 9.87 (P1) aabaababb aabababab
9.75 (P2) aaababbab aabaabbab aabababab
aabababab aabababab 9.98 (P1) aaabababb
9.76 (P1) aabaababb 9.88 (P1) aabaababb aaabbabab
aabaabbab aababaabb aabababab
9.77 (P1) aabaababb aabababab aabababab
aabaabbab 9.89 (P1) aabaababb 9.99 (P1) aaababbab
9.78 (P1) aababbaab aabbaabab aaababbab
aababbabb aabababab aabababab
9.79 (P1) aababaabb 9.90 (P1) aababaabb aabababab
aabbaabab aabbabaab 9.100 (P1) aaabbabab
9.80 (P1) aababbabb aabababab aaabababb
aabbabbab 9.91 (P1) aababaabb aabababab
9.81 (P1) aaabaabbb aabababab aabababab
aabaababb aababbaab 9.101 (P1) aaabbabab
aabababab 9.92 (P1) aababbaab aaabababb
aabbabaab aabababab
aabababab aabababab
Appendix B. Number of automorphic conjugacy classes of each type

This table gives the number of automorphic conjugacy classes whose minimal
words have length n for each graph type in Theorems 2.1, 2.2, and 2.3.
n (P1) (P2) (P3) (R1) (R2) (R3) (R4) (R5) (R6) (R7)
0 0 0 0 0 0 0 1 0 0 0
1 0 0 1 0 0 0 0 0 0 0
2 0 0 1 0 0 0 0 0 0 0
3 0 0 1 0 0 0 0 0 0 0
4 0 0 1 0 0 0 1 1 0 0
5 0 1 3 0 0 0 0 0 0 0
6 4 0 6 0 0 0 0 0 0 0
7 10 1 5 0 0 0 0 0 0 0
8 22 0 8 1 2 3 1 3 1 2
9 81 5 15 0 0 0 0 0 0 0
10 298 4 38 0 0 0 0 0 0 0
11 855 7 49 0 0 0 0 0 0 0
12 2140 4 96 4 12 244 1 7 5 31
13 7040 29 155 0 0 0 0 0 0 0
14 22244 30 342 0 0 0 0 0 0 0
15 64774 49 553 0 0 0 0 0 0 0
16 175209 46 1104 11 70 10899 1 19 15 380
17 543631 185 1927 0 0 0 0 0 0 0
18 1649842 232 3892 0 0 0 0 0 0 0
19 4824825 343 6889 0 0 0 0 0 0 0
20 13535352 406 13592 35 400 473355 1 55 51 4547
Appendix C. Number of paths of each size

The following table gives the number of (P1) classes W whose minimal words
have length n and whose graph Γ(W ) has m vertices. Zeros are omitted.
n m=1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
0
1
2
3
4
5
6 4
7 10
8 22
9 35 26 15 5
10 224 35 22 12 5
11 741 44 33 20 12 5
12 1984 53 40 29 17 12 5
13 4538 1964 401 76 27 17 12 5
14 17064 3762 1052 236 72 24 17 12 5
15 55096 6433 2279 633 205 70 24 17 12 5
16 158613 10156 4197 1440 477 201 67 24 17 12 5
17 415072 110789 12916 3041 1043 446 199 67 24 17 12 5
18 1353447 250705 35075 6714 2250 888 442 196 67 24 17 12 5
19 4197308 513440 89404 16198 4995 1862 857 440 196 67 24 17 12 5
20 12303132 968489 204968 40097 11122 4226 1707 853 437 196 67 24 17 12 5
The following tables give the number of (P2) (left table) and (P3) (right ta-
ble) classes W whose minimal words have length n and whose graph Γ(W ) has m
vertices.
n m=1 2 3 4 5 6 7 8 9 n m=1 2 3 4 5 6 7 8 9 10
0 0
1 1 1
2 2 1
3 3 1
4 4 1
5 0 1 5 3
6 6 5 0 1
7 0 0 1 7 5
8 8 7 0 0 1
9 2 2 0 1 9 15
10 2 2 10 31 4 2 0 1
11 2 2 2 0 1 11 49
12 2 2 12 85 4 4 2 0 1
13 18 6 2 2 0 1 13 155
14 22 6 2 14 301 28 8 2 2 0 1
15 26 12 6 2 2 0 1 15 553
16 30 14 2 16 1031 44 16 8 2 2 0 1
17 138 26 10 6 2 2 0 1 17 1927
18 188 36 6 2 18 3659 172 38 12 6 2 2 0 1
19 242 58 22 10 6 2 2 0 1 19 6889
20 308 82 14 2 20 13123 336 82 28 12 6 2 2 0 1
Acknowledgement
We thank the referee for several good suggestions.
References
[1] Bobbe Cooper and Eric Rowland, Growing words in the free group on two generators, Illinois
J. Math. 55 (2011), no. 2, 417–426 (2012). MR3020689
[2] P. J. Higgins and R. C. Lyndon, Equivalence of elements under automorphisms of a free group,
J. London Math. Soc. (2) 8 (1974), 254–258. MR0340420 (49 #5175)
[3] Bilal Khan, The structure of automorphic conjugacy in the free group of rank two, Compu-
tational and experimental group theory, Contemp. Math., vol. 349, Amer. Math. Soc., Provi-
dence, RI, 2004, pp. 115–196, DOI 10.1090/conm/349/06360. MR2077762 (2005f:20066)
[4] Roger C. Lyndon and Paul E. Schupp, Combinatorial group theory, Springer-Verlag, Berlin-
New York, 1977. Ergebnisse der Mathematik und ihrer Grenzgebiete, Band 89. MR0577064
(58 #28182)
[5] Alexei G. Myasnikov and Vladimir Shpilrain, Automorphic orbits in free groups, J. Algebra
269 (2003), no. 1, 18–27, DOI 10.1016/S0021-8693(03)00339-9. MR2015300 (2004j:20051)
[6] J. H. C. Whitehead, On Certain Sets of Elements in a Free Group, Proc. London Math. Soc.
S2-41, no. 1, 48, DOI 10.1112/plms/s2-41.1.48. MR1575455
[7] J. H. C. Whitehead, On equivalent sets of elements in a free group, Ann. of Math. (2) 37
(1936), no. 4, 782–800, DOI 10.2307/1968618. MR1503309
School of Mathematics, University of Minnesota, Minneapolis, Minnesota 55455
Laboratoire de combinatoire et d’informatique mathématique, Université du Québec

à Montréal, Montréal, QC H2X 3Y7, Canada
Current address: Université de Liège, Département de Mathématiques, Grande Traverse 12
(B37), 4000 Liège, Belgique
Volume 633, 2015
On elementary free groups
Benjamin Fine, Anthony Gaglione, Gerhard Rosenberger,

and Dennis Spellman
Abstract. An elementary free group G is a group with exactly the same
first order theory as a nonabelian free group. The primary non-free examples
of such groups are orientable surface groups Sg of genus g ≥ 2 and nonori-
entable surface groups Ng of genus g ≥ 4.That these groups are elementary
free provides a powerful tool to prove results in surface groups using the so-
lution to the Tarski problem that otherwise are very difficult. In this paper
we consider and prove properties in elementary free groups some of which
are not first order. These include that all finitely generated elementary free
groups are hyperbolic, stably hyperbolic and are Turner groups, that is satisfy
Turner’s retract theorem for test elements. Further all elementary free groups
are conjugacy separable, have tame automorphism groups and have faithful
two dimensional representations in P SL(2, C).
1. Introduction
As a by-product of the positive solution of the Tarski conjectures by Khar-
lampovich and Myasnikov [46]–[50] and Sela [71]–[76] it was proved that the class
of non-free groups that have exactly the same first order theory as the class of
nonabelian free groups was nonempty. Such groups are called elementary free
groups (or elementarily free groups) and both sets of authors provide complete
characterizations of the finitely generated instances of them. In the Kharlampovich-
Myasnikov approach these are the special NTQ-groups (see [50]). The primary
examples of such groups are the orientable surface groups Sg of genus g ≥ 2 and
the nonorientable surface groups Ng of genus g ≥ 4. That these groups are elemen-
tary free provides a powerful tool to prove some results in surface groups that are
otherwise quite difficult. For example J.Howie [41] and independently O. Bogopol-
ski and O. Bogopolski and K.Sviridov [9], [10] proved that a theorem of Magnus
about the normal closures of elements in free groups holds also in surface groups
of appropriate genus (see section 3). Their proofs were nontrivial. However it was
proved (see [19] and [38]) that this result is first order and hence automatically
true in any elementary free group. In [19] a large collection of such results was
given. Such results were called something for nothing results. Of course any such
first order result true in a nonabelian free group must hold in any elementary free
2010 Mathematics Subject Classification. Primary 20F67; Secondary 20F65, 20E06, 20E07.
Key words and phrases. Tarski problems, elementary free groups, hyperbolic group, limit
group, conjugacy separable.
2015
41
42 B. FINE, A. GAGLIONE, G. ROSENBERGER, AND D. SPELLMAN
group. However elementary free groups satisfy many other properties beyond first
order results and this is what we examine in the present paper.
A finitely generated elementary free group G must be a limit group (see sec-
tion 3) and many of its properties follow from the structure theory of limit groups.
Hence such a group must be CSA and any 2-generator subgroup is either free or
abelian. Further we prove that an elementary free group has cyclic centralizers.
This is not a first order statement, however from this we get that if two elements
commute in a finitely generated elementary free group then they are both pow-
ers of a single element. This is not true in a general elementary free group and
we give an example. From the cyclic centralizer property we can obtain that a
finitely generated elementary free group must be hyperbolic, stably hyperbolic and
a Turner group, that is the test elements, if there are any, in any finitely generated
elementary free group are precisely those elements that do not lie in any proper
retract. Further we can prove that any finitely generated elementary free group
is conjugacy separable and hence has a solvable conjugacy problem. Further, the
automorphism group of a finitely generated elementary free group is tame. Finally
any elementary free group has a faithful constructible representation in P SL(2, C).
In the next section we give the necessary background material.
2. The Tarski Problems and Elementary Free Groups

Alfred Tarski in 1940 made three well-known conjectures concerning nonabelian
free groups. We call these the Tarski Problems or Tarski Conjectures and they
asked, among other things, whether all nonabelian free groups satisfy the same first-
order or elementary theory.
Recall that a first-order sentence in group theory has logical symbols ∀, ∃, ∨,
∧, ∼ but no quantification over sets. A first-order theorem in a free group is a
theorem that says a first-order sentence is true in all nonabelian free groups. We
make this a bit more precise:
We start with a first-order language appropriate for group theory. This lan-
guage, which we denote by L0 , is the first-order language with equality containing
a binary operation symbol · a unary operation symbol −1 and a constant symbol
1. A universal sentence of L0 is one of the form ∀x{φ(x)} where x is a tuple of
distinct variables, φ(x) is a formula of L0 containing no quantifiers and containing
at most the variables of x. Similarly an existential sentence is one of the form
∃x{φ(x)} where x and φ(x) are as above. A universal-existential sentence is
one of the form ∀x∃y{φ(x, y)}. Similarly defined is an existential-universal sen-
tence. It is known that every sentence of L0 is logically equivalent to one of the
form Q1 x1 ...Qn xn φ(x) where x = (x1 , ..., xn ) is a tuple of distinct variables, each
Qi for i = 1, ..., n is a quantifier, either ∀ or ∃, and φ(x) is a formula of L0 con-
taining no quantifiers and containing free at most the variables x1 , ..., xn . Further
vacuous quantifications are permitted. Finally a positive sentence is one logically
equivalent to a sentence constructed using (at most) the connectives ∨, ∧, ∀, ∃.
If G is a group then the universal theory of G consists of the set of all
universal sentences of L0 true in G. We denote the universal theory of a group
G by T h∀ (G). Since any universal sentence is equivalent to the negation of an
existential sentence it follows that two groups have the same universal theory if
and only if they have the same existential theory. The set of all sentences of L0
true in G is called the first-order theory or the elementary theory of G. We
ON ELEMENTARY FREE GROUPS 43
denote this by T h(G). We note that being first-order or elementary means that
in the intended interpretation of any formula or sentence all of the variables (free
or bound) are assumed to take on as values only individual group elements - never,
for example, subsets of, nor functions on, the group in which they are interpreted.
We say that two groups G and H are elementarily equivalent (symbolically
G ≡ H) if they have the same first-order theory, that is T h(G) = T h(H).
Group monomorphisms which preserve the truth of first-order formulas are
called elementary embeddings. Specifically, if H and G are groups and
f :H →G
is a monomorphism then f is an elementary embedding provided whenever
φ(x0 , ..., xn ) is a formula of L0 containing free at most the distinct variables x0 , ..., xn
and (h0 , ..., hn ) ∈ H n+1 then φ(h0 , , ..., hn ) is true in H if and only if
φ(f (h0 ), , ..., f (hn ))
is true in G. If H is a subgroup of G and the inclusion map i : H → G is an
elementary embedding then we say that G is an elementary extension of H.
Two very important concepts in the elementary theory of groups, are com-
pleteness and decidability. Given a nonempty class of groups X closed under
isomorphism we say that its first-order theory is complete if given a sentence φ
of L0 either φ is true in every group in X or φ is false in every group in X . The
first-order theory of X is decidable if there exists a recursive algorithm which,
given a sentence φ of L0 , decides whether or not φ is true in every group in X .
The positive solution to the Tarski Problems, given by Kharlampovich and
Myasnikov (see [46]–[54] and independently by Sela (see [71]–[76]) is given in the
next three theorems:
Theorem 2.1 (Tarski 1). Any two nonabelian free groups are elementarily
equivalent. That is any two nonabelian free groups satisfy exactly the same first-
order theory.
Theorem 2.2 (Tarski 2). If the nonabelian free group H is a free factor in the
free group G then the inclusion map H → G is an elementary embedding.
In addition to the completeness of the theory of the nonabelian free groups
the question of its decidability also arises. The decidability of the theory
of nonabelian free groups means the question of whether there exists a recursive
algorithm which, given a sentence φ of L0 , decides whether or not φ is true in every
nonabelian free group. Kharlampovich and Myasnikov, in addition to proving the
two above Tarski conjectures also proved the following.
Theorem 2.3 (Tarski 3). The elementary theory of the nonabelian free groups
is decidable.
Prior to the solution of the Tarski problems, it was asked whether there exist
non-free elementary free groups. By this it was meant that if all countable
nonabelian free groups do have the same first-order theory do there exist non-
free groups with exactly the same first-order theory as the class of nonabelian free
groups. The answer was yes, and both the Kharlampovich-Myasnikov solution
and the Sela solution provide a complete characterization of the finitely generated
elementary free groups. In the Kharlampovich-Myasnikov formulation these are
given as a special class of what are termed NTQ groups (see [46]–[50]). The primary
examples of non-free elementary free groups are the orientable surface groups of
genus g ≥ 2 and the nonorientable surface groups of genus g ≥ 4. Recall that a
surface group is the fundamental group of a compact surface. If the surface is
orientable it is an orientable surface group otherwise a nonorientable surface group.
If Sg denotes the orientable surface group of genus g, then Sg has a one-relator
presentation with a quadratic relator.
Sg = a1 , b1 , ..., ag , bg ; [a1 , b1 ]...[ag , bg ] = 1
.
Groups with presentatations similar to this play a major role in the structure
theory of fully residually free groups and NTQ groups (see [46]–[51]).
Further if Ng denotes the nonorientable surface group of genus g then Ng has
a one-relator presentation with a quadratic relator.
Ng = a1 , ..., ag ; a21 · · · a2g = 1
.
We note that the solution to the Tarski Problems implies that any first-order
theorem holding in the class of nonabelian free groups must also hold in most
surface groups. In many cases proving these results directly is very nontrivial.
Theorem 2.4 (see [46]–[54], [71]–[76]). An orientable surface group of genus
g ≥ 2 is elementary free, that is has the same elementary theory as the class of
nonabelian free groups. Further the nonorientable surface groups Ng for g ≥ 4 are
also elementary free.
We need several other concepts. Let X be a class of groups. Then a group G
is residually X if given any nontrivial element g ∈ G there is a homomorphism
φ : G → H where H is a group in X such that φ(g) = 1. A group G is fully
residually X if given finitely many nontrivial elements g1 , ..., gn in G there is a
homomorphism φ : G → H, where H is a group in X , such that φ(gi ) = 1 for all
i = 1, ..., n. Fully residually free groups have played a crucial role in the study of
equations and first-order formulas over free groups. In Sela’s solution to the Tarski
problems finitely generated fully resiudally free groups are called limit groups.
The universal theory of a group G consists of all universal sentences true in G.
All nonabelian free groups share the same universal theory and a group G is called
universally free if it shares the same universal theory as the class of nonabelian
free groups.
A group G is commutative transitive or CT if commutativity is transitive
on the set of nontrivial elements of G. That is if [x, y] = 1 and [y, z] = 1 for
nontrivial elements x, y, z ∈ G then [x, z] = 1. A subgroup H of a group G is
malnormal if x−1 Hx ∩ H = {1} if x ∈ / H. A group G is CSA if maximal abelian
subgroups are malnormal. CSA implies commutative transitivity but there exist
CT groups that are not CSA. For example it can be shown that a noncyclic one-
relator group G with torsion is CT but not CSA if G has elements of order 2 (see
[22]. Another example of a CT group that is not CSA is the infinite dihedral group
G = a, b; a2 = b2 = 1
. It is straightforward that free products of abelian groups
are CT and hence G is CT. On the other hand the commutator subgroup G is
the cyclic subgroup of G generated by ab. A nonabelian CSA group cannot have a
nontrivial abelian normal subgroup and hence G is not CSA.
Remeslennikov [66] and independently Gaglione and Spellman [36] proved the
following remarkable theorem which became one of the cornerstones in the proof of
the Tarski problems (see [50] and [71]–[76].)
Theorem 2.5. Suppose G is nonabelian and residually free. Then the following
are equivalent:
(1) G is fully residually free,
(2) G is commutative transitive,
(3) G is universally free.
Therefore the class of nonabelian fully residually free groups coincides with the
class of residually free universally free groups. The equivalence of (1) and (2) in the
theorem above was proved originally by Benjamin Baumslag [1], where he intro-
duced the concept of fully residually free. Any finitely generated elementary free
group being universally free must satisfy this theorem and hence be fully residually
free.
In [14] classes of groups X were studied for which being fully residually X
is equivalent to being residually X and commutative transitive, thus extending
Baumslag’s result.
3. Surface Groups and Magnus’ Theorem

Magnus proved the following theorem about the normal closures of elements in
nonabelian free groups:
Theorem 3.1 (Magnus). Let F be a nonabelian free group and R, S ∈ F .
Then if N (R) = N (S),it follows that R is conjugate to either S or S −1 . Here N (g)
denotes the normal closure in F of the element g.
J. Howie [41] and independently O. Bogopolski [9] and Bogopolski and V.
Sviridov [10] gave a proof of this for surface groups. Howie’s proof was for orientable
surface groups while Bogopolski and Sviridov also handled the nonorientable case.
That is Magnus’s theorem holds if the free group F is replaced by a surface group
of approrpiately high genus. Their proofs were nontrivial and Howie’s proof used
the topological properties of surface groups. Howie further developed, as part of
his proof of Magnus’ theorem for surface groups, a theory of one-relator surface
groups. These are surface groups modulo a single additional relator. Bogopolski
and Bogopolski-Sviridov proved in addition that Magnus’s Theorem holds in even
a wider class of groups.
With some work it can be determined that Magnus’ result is actually a first-
order theorem on nonabelian free groups and hence from the theorems concerning
the solution of the Tarski problems it holds automatically in all elementary free
groups. In particular Magnus’ theorem will hold in surface groups, both orientable
and nonorientable of appropriate genus. If G is a group and g ∈ G then N (g), as
in the statement of Magnus’s Theorem above, will denote the normal closure in G
of the element g.
Theorem 3.2. Let G be an elementary free group and R, S ∈ G. Then if
N (R) = N (S) it follows that R is conjugate to either S or S −1 .
Before exhibiting the proof of this result we mention the following two corol-
laries which extend Magnus’s Theorem to surface groups and recover the results of
Howie [41], Bogopolski [9] and Bogopolski-Sviridov [10].
Corollary 3.1 ([41], [9]). Let Sg be an orientable surface group of genus
g ≥ 2. Then Sg satisfies Magnus’s theorem, that is if u, v ∈ Sg and N (u) = N (v)
it follows that u is conjugate to either v or v −1 .
Corollary 3.2 ([10]). Let Ng be a nonorientable surface group of genus g ≥ 4.

Then Ng satisfies Magnus’s theorem, that is if u, v ∈ Ng and N (u) = N (v) it follows
that u is conjugate to either v or v −1 . The genus g ≥ 4 is essential here.
We now present a proof of Theorem 3.2. From Theorem 3.2 the two corollaries
describing this result in surface groups follow easily based on the solution to the
Tarski problems coupled with the facts that orientable surface groups of genus g ≥ 2
and nonorientable surface gorups of genus g ≥ 4 are elementary free.
Proof of Theorem 3.2. We show that Magnus’s theorem is actually a first-
order result in nonabelian free groups. Since it is known to be true in nonabelian
free groups it will then from the solution to the Tarski problems be true in any
elementary free group.
Magnus’s theorem can be given by a sequence of elementary sentences of the
form (see also [38]).
{∀R, S ∈ G, ∀g ∈ G∃g1 , ..., gt , h1 , ..., hk }
(g −1 Rg = g1−1 S ±1 g1 ...gt−1 S ±1 gt ) ∧ (g −1 Sg = h−1

1 R
±1
h1 ...h−1
k R
±1
hk )
=⇒ {∃x ∈ G(x−1 Rx = S ∨ x−1 Rx = S −1 )}.

Magnus’s theorem is therefore a first-order result and the theorem follows.

As described prior to the proof it follows that any elementary free group and
hence surface groups of the appropriate genus satisfy Magnus’s theorem. This
recovers the results in [41], [10], [9]. Actually more is true. An examination of
the sentences capturing that Magnus’s theorem (Theorem 3.1) is first-order shows
that the sentences are universal-existential. Hence the theorem holds in the almost
locally free groups of Gaglione and Spellman [37].
4. Cyclic Centralizers and Commuting Elements

In nonabelian free groups, nontrivial elements have cyclic centralizers. It follows
that if two elements in a nonabelian free group commute then they must be powers
of a single element. This result however is not first order.
The result is given by the sentence
∀{x, y ∈ F }([x, y] = 1) =⇒ ∃{w ∈ F }∃{{m, n ∈ Z}}(x = wm ∧ y = wn ).
Since we must quantify over the integers, which are not included in the language
L0 , this is not first-order in the language of group theory.
However in the case of finite generation, the fact that elementary free groups
have cyclic centralizers can be proved directly. From this we get that if two elements
commute in a finitely generated elementary free group then they are both powers
of a single element.
Theorem 4.1. Let G be a finitely generated elementary free group. Then G
has cyclic centralizers of nontrivial elements. It follows that if x, y ∈ G and x,y
commute then both x and y are powers of a single element w ∈ G.
Proof. Let G be a finitely generated elementary free group. Then G is finitely

generated and fully residually free. It follows from the fact that finitely generated
fully residually free groups are commutative transitive that G has abelian centraliz-
ers. Applying Szmielew’s criteria for elementary equivalence of abelian groups(see
[78]) it follows that in any elementary free group the centralizer of any nontrivial
element is elementarily equivalent to the infinite cyclic group. In particular such
centralizers must satisfy the following sentences:
∀x1 , x2 (x1 x2 = x2 x1 )
∃x(x = 1);
for each integer n ≥ 2 the sentence
∀x((xn = 1) → (x = 1))
and the sentence
∀x1 , x2 , x3 ∃y((x1 x−1 −1 −1
2 = y ) ∨ (x1 x3 = y ) ∨ (x2 x3 = y ))
2 2 2
asserting that, modulo 2, there are at most 2 distinct elements.

A result of Gaglione, Lipschutz and Spellman (Lemma 3.6 in [38]) shows that
up to isomorphism the only finitely generated group M which can satisfy these
properties simultaneously is the infinite cyclic group. Here we will repeat the proof
given there.
Suppose not and M is a finitely generated abelian group satisfying the above
sentences. Then M contains a rank 2 free abelian direct factor A and suppose that
M = A × B.
Now let (a1 , a2 , a3 ) ∈ A3 . Then there is a ∈ A, b ∈ B such that
a1 a−1 −1 −1
2 = a b ∨ a1 a3 = a b ∨ a2 a3 = a b .
2 2 2 2 2 2
Since the product is direct b2 = 1 is the only possibility. Then, writing A(X 2 ) for
the subgroup of A generated by the squares, a1 ≡ a2 mod A(X 2 ) or a1 ≡ a3 mod
A(X 2 ) or a2 ≡ a3 mod A(X 2 ). Since (a1 , a2 , a2 ) ∈ A3 was arbitrary, the index
[A : A(X 2 )] ≤ 2. However if A has rank 2 it follows that [A : A(X 2 )] = 4. This
contradiction shows that M is cyclic.
We mention that Theorem 4.1 is not true in general elementary free groups. As
an example, let D be a nonprincipal ultrafilter on Z (see [7]). Let F = a1 , a2 ;
the
free group of rank 2 on a1 , a2 and let ∗ F = F Z /D be the corresponding ultrapower
so that ∗ F is elementary free (see [7]). Consider the elements
[(a1 )k∈Z ]D = [(..., a1 , a1 ....a1 , ...)]D
and
[(ak1 )k∈Z ]D = [(..., a−2 −1 2
1 , a1 , 1, a1 , a1 ...)]D .
These commute but there is no fixed element B of which they are both powers.
We note that any result in a finitely generated elementary free group must hold
automatically in surface groups of appropriate genus. As a corollary we get that
the above commuting result must be true in surface groups a fact that can also be
obtained directly from the amalgam structure of such groups or from their faithful
representations as discrete subgroups of P SL(2, C).
Corollary 4.1. Let G be either an orientable surface group of genus g ≥ 2 or

a nonorientable surface group of genus g ≥ 4. If x, y ∈ G and x,y commute then
both x and y are powers of a single element w ∈ F .
We next give some examples that are less trivial but not obvious in a surface
group. First we need the following theorem that can be easily proved in free groups.
Theorem 4.2. Let F be a free group and n, k nonzero integers. For all x, y ∈ F
if [xn , y] = [x, y k ] then either n = k = 1 or x, y commute and both are powers of a
single element.
The first part of the result that either n = k = 1 or [x, y] = 1 is first-order
given by a sequence of elementary sentences, one for each (n, k) ∈ Z2 \ {(1, 1)};
∀x, y ∈ F ([xn , y] = [x, y k ]) =⇒ [x, y] = 1.
Therefore this part of the result must hold in any elementary free group. Further
if the elementary free group is finitely generated the second part must also hold.
Corollary 4.2. Let G be an elementary free group. If x, y ∈ G and if [xn , y] =
[x, y k ] then either n = k = 1 or x, y commute. If G is finitely generated then both
x and y are powers of a single element w ∈ G.
Again this result can be applied to surface groups since these are are finitely
generated.
Corollary 4.3. Let G be either an orientable surface group of genus g ≥ 2
or a nonorientable surface group of genus g ≥ 4. If x, y ∈ G and if [xn , y] = [x, y k ]
then either n = k = 1 or x, y commute and then both x and y are powers of a single
element w ∈ G.
Csorgo, Fine and Rosenberger [15] proved the following extension of this.
Theorem 4.3 ([14]). Suppose F is a nonabelian free group and x, y, u, v ∈ F
with [x, y] = 1 and u, v in the subgroup generated by x, y. Then if [x, y] is conjugate
to a power of [u, v] within x, y
that is there exists a k with [x, y] = g([u, v]k )g −1 for
some g ∈ x, y
and [x, y m ] = [u, v n ] it follows that m = n. Further if m = n ≥ 2
then y is conjugate within x, y
to v or v −1 .
As with Magnus’s theorem this can be shown to be given by a sequence of
first-order sentences and is hence a first-order result. Therefore this holds in any
elementary free group.
Theorem 4.4. Let G be an elementary free group and x, y, u, v ∈ G with [x, y] =
1 and u, v in the subgroup generated by x, y. Then if [x, y] is conjugate to a power
of [u, v] within x, y
that is there exists a k with [x, y] = g([u, v]k )g −1 for some
g ∈ x, y
and [x, y m ] = [u, v n ] it follows that m = n. Further if m = n ≥ 2 then y
is conjugate within x, y
to v or v −1 .
In particular we get the extension to surface groups.
Corollary 4.4. Let G be either an orientable surface group of genus g ≥ 2 or
a nonorientable surface group of genus g ≥ 4 and suppose that x, y, u, v ∈ G with
[x, y] = 1 and u, v in the subgroup generated by x, y. Then if [x, y] is conjugate to
a power of [u, v] within x, y
that is there exists a k with [x, y] = g([u, v]k )g −1 for
some g ∈ x, y
and [x, y m ] = [u, v n ] it follows that m = n. Further if m = n ≥ 2
then y is conjugate within x, y
to v or v −1 .
5. Hyperbolicity and Stable Hyperbolicity

Hyperbolic groups are finitely presented groups that have become a funda-
mental structure in geometric group theory. Finitely generated free groups are
hyperbolic. Roughly a group G is hyperbolic if the geometry of the Cayley graph
of G is hyperbolic in the sense that it satisfies the δ-hyperbolic property. Torsion-
free hyperbolic groups have cyclic centralizers for nontrivial elements and it has
been proved that a limit group G (finitely generated fully residually free group)
is hyperbolic if and only if in G the centralizer of a nontrivial element is cyclic
(see [50]). A finitely generated elementary free group, being also universally free,
is also fully residually free (by the Gaglione-Spellman-Remeslennikov result) and
from Theorem 4.1 such a group also has the cyclic centralizer property. Thus we
have proved that any finitely generated elementary free group must be hyperbolic.
Theorem 5.1. Let G be a finitely generated elementary free group. Then G is
hyperbolic.
A group G is stably hyperbolic if G is hyperbolic and for any endomorphism
φ : G → G for all n there is an m ≥ n such that φm (G) is hyperbolic. Using the
cyclic centralizer result we obtain.
stably hyperbolic.
Proof. Let G be a finitely generated elementary free group. As in the proof of
the last theorem G is finitely generated and fully residually free. Since G is finitely
generated and elementary free it has cyclic centralizers and is therefore hyperbolic.
Now let φ : G → G be an endomorphism. Since G is finitely generated then φn (G)
is also a finitely generated fully residually free group for any natural number n. The
property of having cyclic centralizers holds in any subgroup and therfore φn (G) is
hyperbolic for any n. Therefore G is stably hyperbolic.
6. The Retract Theorem and Turner Groups

An element g in a group G is a test element if whenever f (g) = g for some
endomorphism of G then f must be an automorphism. This concept dates back to
Nielsen who showed that [x, y] is a test element in the free group on {x, y}. Test
elements in a free group are called test words (see [35]).
Turner [79] gave the following characterization of test words in finitely gen-
erated free groups. This is now referred to as either the Retract Theorem or
Turner’s Theorem.
Theorem 6.1. Let F be a finitely generated nonabelian free group. Then an
element g ∈ F is a test word if and only if g lies in no proper retract.
The question whether Turner’s theorem is first-order or not was considered in
[20]. We call an element g in a group G nonprojectible if it lies in no proper retract
of G. We then call a group G a Turner group if for g ∈ G being nonprojectible
in G implies that g is a test element. Equivalently G is a Turner group if and only
if the Retract Theorem holds. Hence Turner’s theorem says that nonabelian free
groups are Turner groups.
A result of O’Neill and Turner (see [63]) shows that stably hyperbolic groups
are Turner groups. From Theorem 5.2 it then follows then that finitely generated
elementary free groups are Turner groups, that is they satisfy the Retract Theorem.

a Turner group, that is G satisfies the Retract Theorem and hence the test elements
in G are precisely those elements that avoid any proper retract.
Proof. Let G be a finitely generated elementary free group. From Theorem
5.2 G is stably hyperbolic. Therefore G satisfies the Retract Theorem from the
result of O’Neill and Turner.
As in the previous cases this then extends to surface groups of appropriate

genus.
or a nonorientable surface group of genus g ≥ 4. Then G is a Turner group.
In [63] it was proved directly that there are test elements in surface groups.
However this also follows directly from the previous corollary since not every ele-
ment in either Sg or Ng falls in a proper retract.
or a nonorientable surface group of genus g ≥ 4. Then G has test elements.
In [20] the following results were proved showing that Turner’s Theorem is not
first-order and not the model class of any set of sentences of L0 .
Theorem 6.3 (Nondefinability Theorem). (1) There is no set N (x) of formulas
of L0 such that, for an arbitrary group G and arbitrary element g ∈ G , N (g) holds
if and only if g is nonprojectible.
(2) There is no set T (x) of formulas of L0 such that, for an arbitrary group
G and arbitrary element g ∈ G, T (g) holds if and only if g is a test element.
Theorem 6.4 (Nonaxiomatizability Theorem). The class of Turner groups is
not the model class of any set of sentences of L0 .
7. Conjugacy Separability of Elementary Free Groups

A group G is conjugacy separable if given any two elements g1 , g2 ∈ G either
g1 is conjugate to g2 or there exists a homomorphism ρ : G → H where H is a finite
group and in which ρ(g1 ) is not conjugate to ρ(g2 ). It is known that all free groups
are conjugacy separable. Here we next prove that all finitely generated elementary
free groups are conjugacy separable.
conjugacy separable.
Proof. Suppose G is an finitely generated elementary free group and g1 , g2 are
two nonconjugate elements of G. Since free groups are conjugacy separable to show
that G is conjugacy separable it suffices to show that there is a free homomorphic
image of G in which the images of g1 and g2 are nonconjugate.
Suppose there is no free homomorphic image of G in which g1 is not conjugate
to g2 . Note that a finitely generated elementary free group, in fact more generally
a finitely generated fully residually free group must be finitely presented (see [47]).
Fix a finite presentation for G,
a1 , ..., an ; R1 (a1 , ..., an ) = ... = Rm (a1 , ..., an ) = 1
and suppose that gi = wi (a1 , ..., an ) for i = 1, 2. Then since there are no free
homomorphic images of G in which g1 and g2 are not conjugate the following
universal-existential sentence which we denote by 1
of L0 would be true in every
nonabelian free group
−1
∀x1 , ..., xn ∃y(∧m
i=1 (Ri (x1 , .., xn ) = 1)) → (w2 (x1 , ..., xn ) = y w1 (x1 , ..., xn )y).
It follows that 1
would have to be true in G. But this contradicts the fact
that g1 is not congugate to g2 in G. Therefore there must exist a free homomorphic
image in which g1 and g2 are not conjugate and hence G is conjugacy separable.
8. Tame Automorphisms of Elementary Free Groups

As part of the proof of the Tarski theorems, both Kharlampovich-Myasnikov
and Sela, completely described the structure of finitely generated fully residually
free groups or limit groups in terms of what is called the JSJ-decomposition. These
structure results can be used to both solve the isomorphism problem for limit groups
and to prove that the automorphism group of a finitely generated fully residually
free group is tame. It follows that the automorphism group of an elementary free
group is also tame. We explain these concepts.
A minimal finite presentation of a finitely presented group G is a presentation
that is minimal with respect to the number of generators. Hence a presentation
G = x1 , ..., xn ; r1 , ..., rm
is a minimal finite presentation for G if n = rank(G),
the minimal number of generators necessary to present G. Now suppose that
G = x1 , ..., xn ; r1 , ..., rm
with 1 ≤ n, m < ∞ is minimal finite presentation of
G. Let F = x1 , ..., xn ;
be the free group of rank n on {x1 , ..., xn }. An auto-
morphism α : G → G is tame if it is induced by or lifts to an automorphism
on F ( considered as free on the generators of G). If each automorphism of G
is tame we say that the automorphism group Aut(G) is tame. In [77] Shpilrain
gives a survey of some of the known general results on tame automorphisms and
tame automorphism groups. If G is a surface group a result of Zieschang [81] and
improved upon by Rosenberger [70] shows that G has only one Nielsen class of
minimal generating systems. An easy consequence of this is that that Aut(G) is
tame. Rosenberger (see [60] or [69]) uses the term almost quasifree for a finitely
presented group which has a tame automorphism group. If G is almost quasifree,
G = x1 , ..., xn ; r1 , ..., rm
, 1 ≤ n, m < ∞ is a minimal finite presentation of G and,
in addition, each automorphism of F = x1 , ..., xn ;
induces an automorphism of G,
G is called quasifree. Rosenberger observed that a non-cyclic, non-free one-relator
group is quasifree only if it has a presentation a, b; [a, b]n = 1
for n ≥ 1. This is a
Fuchsian group if n ≥ 2 and isomorphic to a free abelian group of rank 2 if n = 1.
JSJ decompositions were introduced by Rips and Sela [67]. A JSJ-decom-
position of a group G is a graph of groups decomposition of G with abelian edge
groups that encodes all other graph of groups decompositions of G. Any finitely gen-
erated fully residually free group has a JSJ decomposition with cyclic edge groups
and vertex groups of specific types if it is not abelian or a surface group. We refer
to the relevant papers for further discussions of these but mention that Bumagin,
Kharlampovich and Myasnikov [12] used the JSJ decomposition to describe the
automorphism group of a limit group. Further specifics can be found in [4].
As an application of the study of automorphisms of fully residually free groups

Bumagin, Kharlampovich and Myasnikov [12] were able to prove that the isomor-
phism problem is solvable for finitely generated fully residually free groups. This is
actually part of the algorithmic study of this class of groups. In particular Khar-
lampovich, Myasnikov, Remeslennikov and Serbin [55] were able to translate the
method of Stallings foldings to fully residually free groups by considering infinite
words in F Z[t] . Doing this they were able to algorithmically solve many problems
in fully residually free groups mirroring the algorithmic solutions in absolutely free
groups (see [55]).
Theorem 8.1 ([12]). The isomorphism problem is solvable in the class of

finitely generated fully residually free groups. That is given two finite presentations
that are known to define fully residually free groups there is an effective algorithm
to determine if the defined groups are isomorphic.
As an additional consequence of the JSJ decomposition of a fully residually

free group and the work of Bumagin, Kharlampovich and Mysasnikov, the tame-
ness of Aut(G) for a limit group was proved by Fine, Kharlmapovich, Myasnikov,
Rosenberger and Remeslennikov [23].
Theorem 8.2 ([23]). The automorphism group Aut(G) of a finitely generated

freely indecomposable fully residually free group G is tame with respect to a presen-
tation for the JSJ decomposition for G.
Since each finitely generated elementary free group is universally free and hence
fully residually free the proof of the corollary is immediate.
Corollary 8.1. The automorphism group of a finitely generated freely inde-

composable elementary free group G is tame.
We note that the converse of this corollary is false. That is there do exist
groups ( in fact hyperbolic groups) where every automorphism is tame but which
are not fully residually free. As an example the groups
G = a1 , ..., an ; aα
1 · · · an
, with n ≥ 3, 2 ≤ α1 , ..., αn
1 αn
and
H = s1 , ..., sn ; s21 , ..., s2n−1 , s2k+1
n , s1 · · · sn
with n = 2, n ≥ 4 even and k ≥ 1,
are all hyperbolic. Further every automorphism is tame (see [68] and [32]). How-
ever not all of these groups are fully residually free.
9. Faithful Representations in P SL(2, C)

It is well-known that the surface groups have faithful representations as dis-
crete subgroups of P SL(2, C) in fact also within P SL(2, R). Fine and Rosenberger
[26]–[27] proved that any finitely generated fully residually free group has a faithful
representation within P SL(2, C). This follows from the fact that the limit groups
are precisely the finitely generated subgroups of non-standard free groups. In the
case where the limit group is hyperbolic, that is has cyclic centralizers, this repre-
sentation can be effectively constructed using the JSJ decomposition.
Theorem 9.1. Let G be a hyperbolic limit group and in particular any finitely
generated elementary free group. Then a faithful representation of
ρ : G → P SL(2, C)
can be effectively constructed from the JSJ decomposition of G.
Using the embedding of a limit group in a nonstandard free group the restriction
to hyperbolicity can be removed.
Theorem 9.2. Any limit group and in particular any finitely generated ele-
mentary free group has a faithful representation in P SL(2, C)
Theorem 9.1 was proved first using faithful representations of amalgams. We
will discuss this below. Subsequently it was realized that the proof is much simpler
(without an explicit construction) using nonstandard free groups.
We need some additional material, Let I be a nonempty set. Let P (I) be the
power set of I. A subset D0 ⊂ P (I) is a proper filter on I provided:
(1) I ∈ D0 ,
(2) A, B ∈ D0 =⇒ A ∩ B ∈ D0 ,
(3) A ∈ D0 and A ⊂ B ⊂ I =⇒ B ∈ D0 ,
(4) ∅ ∈
/ D0 .
A filter D on I is an ultrafilter on I provided it is maximal in the class of
filters on I
Now let I be a nonempty set and D a proper filter on I and let {Gi }i∈I
be a
family of groups indexed by I. Then the relation on the direct product i∈I Gi
defined by f ≡D g provide that
{i ∈ I : f (i) = g(i)} ∈ D
is a congruence relation. A relation on a group G is a congruence relation provided
it is an equivalence relation that preserves the group operation. It follows that the
subset K of the direct product given by

K = {f ∈ Gi : f ≡D 1}
i∈I
actually defines a normal subgroup.

The reduced product i∈I Gi /D is

the quotient of the direct product i∈I Gi
modulo the normal subgroup K = {f ∈ i∈I Gi : f ≡D 1}. If D is an ultrafilter
on I then the reduced product is the ultraproduct of the family {Gi }i∈I modulo
the ultrafilter D on I. If the family
{Gi }i∈I
consists of a single group G so that the direct product is GI then the ultraproduct
is an ultrapower. If F is a free group we call an ultrapower of F a nonstandard
free group. If A is an algebraic object we will denote an ultrapower of it by A∗ .
The following was proved by Remeslennikov (see [46]–[51]).
Theorem 9.3. Any nonabelian limit group can be embedded in a nonstandard
free group F ∗ .
We now consider groups with exponents. R. Lyndon introduced and studied
the free exponential group F Z[t] over the polynomial ring Z[t]. He then proved that
the group F Z[t] is fully residually free. Hence each finitely generated subgroup of
F Z[t] is a limit group. Myasnikov and Remeslennikov [61] axiomatized the notion
of exponential groups and proved that F Z[t] can be obtained starting from F by an
infinite chain of free extensions of centralizers. The basic idea is that to construct
F Z[t] one must extend each centralizer sufficiently many times so that each central-
izer is a free abelian group of infinite rank and hence isomorphic to the additive
group of Z[t]. This then implies that any finitely generated subgroup of F Z[t] is a
subgroup of a group that can be obtained from F by finitely many extensions of
centralizers. Subsequently Kharlampovich and Myasnikov (see [46]–[51]) proved
that a finitely generated group G is fully residually free if and only if it is embed-
dable in F Z[t] . This implies that any limit group can be embedded as a subgroup
of F Z[t] where F = Fω is a free group of countably infinite rank. It follows that
each finitely generated fully residually free group can be obtained as a subgroup
of a group that can be obtained from a free group by finitely many extensions of
centralizers. This was the main idea used in the proof of the faithful representation
of hyperbolic limit groups given in [26]. Subsequently it was proved that the em-
bedding of a given limit group G into F Z[t] is effective. We summarize these and
will use them in our main proof.
Theorem 9.4 (see [46]–[51]). Let G be a finitely generated group and F = Fω
be a free group of countably infinite rank. Then G is fully residually free and hence
a limit group if and only if G can be embedded as a subgroup of F Z[t] .
The final ingredient we need for the main proof is the following.
Lemma 9.1. Any countable free group F can be embedded into both SL(2, C)
and P SL(2, C).
There are many references for this result for example see [27].
Proof of Theorem 9.2 (see [26], [27]). Let G be a limit group. Then G can
be embedded in F ∗ where F ∗ is a nonstandard free group, that is an ultrapower of
F a fixed free group of countable rank.
A finitely generated nonabelian free group F can be embedded into SL(2, Q).
Hence the ultrapower F ∗ can be embedded into the ultrapower SL(2, Q)∗ =
SL(2, Q∗ ). Hence G can be embedded into SL(2, Q∗ ). However since G is finitely
generated it follows that G can be embedded in a finite power of SL(2, Q) and hence
into SL(2, C). We note that it is also true from the same argument that G can be
embedded into SL(2, F) where F is any algebraicially closed field of characteristic
zero but this doesn’t concern us here.
Hence G can be embedded into SL(2, C) and we must show that this faithful
representation can be pushed down to P SL(2, C) = SL(2, C)/Z(SL(2, C) where
Z(SL(2, C) is its center. However the center of SL(2, C) is precisely {I, −I} where
I is the 2 × 2 identity matrix. A limit group is torsion-free so the image of G in
SL(2, C) cannot contain −I. Therefore G embeds into P SL(2, C)
The explicit constructible representation given in Theorem 9.1 depends on the

JSJ decomposition of a limit group. This in turn depends upon faithful P SL(2, C)
representations of two extremely important amalgam constructions: cyclically
pinched one-relator groups and conjugacy pinched one-relator groups.
These constructions are natural algebraic generalization of the one-relator presen-
tation type of a surface group. These types of presentations play a major role in
the structure theory of limit groups and hence of all elementary free groups. A
discussion of the properties of such constructions can be found in [32].
References
[1] B. Baumslag, Residually free groups, Proc. London Math. Soc. (3) 17 (1967), 402–418.
MR0215903 (35 #6738)
[2] G. Baumslag, On generalised free products, Math. Z. 78 (1962), 423–438. MR0140562
(25 #3980)
[3] G. Baumslag, B. Fine, C. F. Miller III, and D. Troeger, Virtual properties of cyclically
pinched one-relator groups, Internat. J. Algebra Comput. 19 (2009), no. 2, 213–227, DOI
10.1142/S0218196709005032. MR2512551 (2010c:20034)
[4] G. Baumslag, A. Myasnikov, and V. Remeslennikov, Discriminating completions of hyperbolic
groups, Geom. Dedicata 92 (2002), 115–143, DOI 10.1023/A:1019687202544. MR1934015
(2003i:20073)
[5] G. Baumslag, A. Myasnikov, and V. Remeslennikov, Algebraic geometry over groups. I. Alge-
braic sets and ideal theory, J. Algebra 219 (1999), no. 1, 16–79, DOI 10.1006/jabr.1999.7881.
MR1707663 (2000j:14003)
[6] G. Baumslag and P. B. Shalen, Amalgamated products and finitely presented groups, Com-
ment. Math. Helv. 65 (1990), no. 2, 243–254, DOI 10.1007/BF02566605. MR1057242
(91j:20071)
[7] J. L. Bell and A. B. Slomson, Models and ultraproducts: An introduction, North-Holland
Publishing Co., Amsterdam-London, 1969. MR0269486 (42 #4381)
[8] M. Bestvina and M. Feighn, A combination theorem for negatively curved groups, J. Differ-
ential Geom. 35 (1992), no. 1, 85–101. MR1152226 (93d:53053)
[9] O. Bogopolski, A surface groups analogue of a theorem of Magnus, Geometric methods in
group theory, Contemp. Math., vol. 372, Amer. Math. Soc., Providence, RI, 2005, pp. 59–69,
DOI 10.1090/conm/372/06874. MR2139677 (2006b:20058)
[10] O. Bogopolski and K. Sviridov, A Magnus theorem for some one-relator groups, The Zi-
eschang Gedenkschrift, Geom. Topol. Monogr., vol. 14, Geom. Topol. Publ., Coventry, 2008,
pp. 63–73, DOI 10.2140/gtm.2008.14.63. MR2484697 (2010k:20064)
[11] A. M. Brunner, R. G. Burns, and D. Solitar, The subgroup separability of free products of
two free groups with cyclic amalgamation, Contributions to group theory, Contemp. Math.,
vol. 33, Amer. Math. Soc., Providence, RI, 1984, pp. 90–115, DOI 10.1090/conm/033/767102.
MR767102 (86e:20033)
[12] I. Bumagin, O. Kharlampovich, and A. Miasnikov, The isomorphism problem for finitely
generated fully residually free groups, J. Pure Appl. Algebra 208 (2007), no. 3, 961–977, DOI
10.1016/j.jpaa.2006.03.025. MR2283438 (2007j:20037)
[13] C. C. Chang and H. J. Keisler, Model theory, 2nd ed., North-Holland Publishing Co.,
Amsterdam-New York-Oxford, 1977. Studies in Logic and the Foundations of Mathemat-
ics, 73. MR0532927 (58 #27177)
[14] L. Ciobanu, B. Fine and G. Rosenberger, Classes of Groups Generalizing a Theorem of
Benjamin Baumslag, to appear in Res. Math.
[15] P. Csörgo, B. Fine, and G. Rosenberger, On certain equations in free groups, Acta Sci. Math.
(Szeged) 68 (2002), no. 3-4, 895–905. Corrected reprint of Acta Sci. Math. (Szeged) 68 (2002),
no. 1-2, 95–105 [ MR1916569 (2003h:20046a)]. MR1954553 (2003h:20046b)
[16] D. J. Collins and H. Zieschang, Combinatorial group theory and fundamental groups, Al-
gebra, VII, Encyclopaedia Math. Sci., vol. 58, Springer, Berlin, 1993, pp. 1–166, 233–240.
MR1265270
[17] J. L. Dyer, Separating conjugates in amalgamated free products and HNN extensions, J.
Austral. Math. Soc. Ser. A 29 (1980), no. 1, 35–51. MR566274 (81f:20033)
[18] B. Fine, A. M. Gaglione, G. Rosenberger, and D. Spellman, n-free groups and questions about
universally free groups, Groups ’93 Galway/St. Andrews, Vol. 1 (Galway, 1993), London
Math. Soc. Lecture Note Ser., vol. 211, Cambridge Univ. Press, Cambridge, 1995, pp. 191–
204, DOI 10.1017/CBO9780511629280.018. MR1342791 (96h:20052)
[19] B. Fine, A. Gaglione, G. Rosenberger and D. Spellman, Something for Nothing: Some Con-
sequences of the Solution to the Tarski Problems, to appear in Groups St. Andrews 2013.
[20] B. Fine, A. Gaglione, S.Lipschutz and D. Spellman, Turner’s Theorem is Not first-order, in
press.
[21] B. Fine, A. M. Gaglione, A. Myasnikov, G. Rosenberger, and D. Spellman, A classification
of fully residually free groups of rank three or less, J. Algebra 200 (1998), no. 2, 571–605,
DOI 10.1006/jabr.1997.7205. MR1610668 (99b:20053)
[22] B. Fine, A. Myasnikov, V. große Rebel, and G. Rosenberger, A classification of conju-
gately separated abelian, commutative transitive, and restricted Gromov one-relator groups,
Results Math. 50 (2007), no. 3-4, 183–193, DOI 10.1007/s00025-007-0245-5. MR2343587
(2008k:20066)
[23] B. Fine, O. G. Kharlampovich, A. G. Myasnikov, V. N. Remeslennikov, and G. Rosenberger,
On the surface group conjecture, Sci. Ser. A Math. Sci. (N.S.) 15 (2007), 1–15. MR2367908
(2009b:20050)
[24] B. Fine, M. Kreuzer and G. Rosenberger, Real Representations of Pinched One-Relator
Groups, to appear.
[25] B. Fine and G. Rosenberger, Algebraic generalizations of discrete groups: A path to com-
binatorial group theory through one-relator products, Monographs and Textbooks in Pure
and Applied Mathematics, vol. 223, Marcel Dekker, Inc., New York, 1999. MR1712997
(2000m:20049)
[26] B. Fine and G. Rosenberger, A note on faithful representations of limit groups, Groups
Complex. Cryptol. 3 (2011), no. 2, 349–355, DOI 10.1515/gcc.2011.014. MR2898897
[27] B. Fine and G. Rosenberger, Faithful representations of limit groups II, Groups Complex.
Cryptol. 5 (2013), no. 1, 91–96. MR3065450
[28] B. Fine, A. Rosenberger, and G. Rosenberger, Quadratic properties in group amalgams, J.
Group Theory 14 (2011), no. 5, 657–671, DOI 10.1515/JGT.2010.069. MR2831964
[29] B. Fine, A. Rosenberger, G. Rosenberger, A Note on Lyndon Properties in One Relator
Groups, Results in Math., 2011, 1-15.
[30] B. Fine, F. Röhl, and G. Rosenberger, Two-generator subgroups of certain HNN groups,
Combinatorial group theory (College Park, MD, 1988), Contemp. Math., vol. 109, Amer.
Math. Soc., Providence, RI, 1990, pp. 19–23, DOI 10.1090/conm/109/1076373. MR1076373
(92c:20041)
[31] B. Fine, F. Röhl, and G. Rosenberger, On HNN-groups whose three-generator subgroups are
free, Infinite groups and group rings (Tuscaloosa, AL, 1992), Ser. Algebra, vol. 1, World Sci.
Publ., River Edge, NJ, 1993, pp. 13–36. MR1377954 (96m:20042)
[32] B. Fine, G. Rosenberger, and M. Stille, Conjugacy pinched and cyclically pinched one-relator
groups, Rev. Mat. Univ. Complut. Madrid 10 (1997), no. 2, 207–227. MR1605642 (99c:20039)
[33] B. Fine, G. Rosenberger, and M. Stille, Nielsen transformations and applications: a survey,
Groups—Korea ’94 (Pusan), de Gruyter, Berlin, 1995, pp. 69–105. MR1476950 (98g:20039)
[34] B. Fine, G. Rosenberger, D. Spellman, and M. Stille, Test words, generic elements and almost
primitivity, Pacific J. Math. 190 (1999), no. 2, 277–297, DOI 10.2140/pjm.1999.190.277.
MR1722895 (2000j:20035)
[35] B. Fine, G. Rosenberger, and M. Stille, Nielsen transformations and applications: a survey,
Groups—Korea ’94 (Pusan), de Gruyter, Berlin, 1995, pp. 69–105. MR1476950 (98g:20039)
[36] A. M. Gaglione and D. Spellman, Even more model theory of free groups, Infinite groups and
group rings (Tuscaloosa, AL, 1992), Ser. Algebra, vol. 1, World Sci. Publ., River Edge, NJ,
1993, pp. 37–40. MR1377955 (96k:20046)
[37] A. M. Gaglione and D. Spellman, Almost locally free groups and the genus question,
Comm. Algebra 26 (1998), no. 9, 2821–2836, DOI 10.1080/00927879808826312. MR1635929
(99i:20003)
[38] A. M. Gaglione, S. Lipschutz, and D. Spellman, Almost locally free groups and a theo-
rem of Magnus: some questions, Groups Complex. Cryptol. 1 (2009), no. 2, 181–198, DOI
10.1515/GCC.2009.181. MR2598987 (2011b:20094)
[39] D. Gildenhuys, O. Kharlampovich, and A. Myasnikov, CSA-groups and sepa-
rated free constructions, Bull. Austral. Math. Soc. 52 (1995), no. 1, 63–84, DOI
10.1017/S0004972700014453. MR1344261 (96h:20053)
[40] C. Gordon and H. Wilton, On surface subgroups of doubles of free groups, J. Lond. Math.
Soc. (2) 82 (2010), no. 1, 17–31, DOI 10.1112/jlms/jdq007. MR2669638 (2011k:20085)
[41] J. Howie, Some results on one-relator surface groups, Bol. Soc. Mat. Mexicana (3) 10 (2004),
no. Special Issue, 255–262. MR2199352 (2006k:20072a)
[42] S. V. Ivanov, On certain elements of free groups, J. Algebra 204 (1998), no. 2, 394–405, DOI
10.1006/jabr.1997.7354. MR1624451 (99e:20035)
[43] A. Juhász and G. Rosenberger, On the combinatorial curvature of groups of F -type and other
one-relator free products, The mathematical legacy of Wilhelm Magnus: groups, geometry
and special functions (Brooklyn, NY, 1992), Contemp. Math., vol. 169, Amer. Math. Soc.,
Providence, RI, 1994, pp. 373–384, DOI 10.1090/conm/169/01667. MR1292912 (95i:20050)
[44] I. Kapovich, P. Schupp, and V. Shpilrain, Generic properties of Whitehead’s algorithm and
isomorphism rigidity of random one-relator groups, Pacific J. Math. 223 (2006), no. 1, 113–
140, DOI 10.2140/pjm.2006.223.113. MR2221020 (2007e:20068)
[45] K. Kearnes, Private e-mail communication.
[46] O. Kharlampovich and A. Myasnikov, Irreducible affine varieties over a free group. I. Irre-
ducibility of quadratic equations and Nullstellensatz, J. Algebra 200 (1998), no. 2, 472–516,
DOI 10.1006/jabr.1997.7183. MR1610660 (2000b:20032a)
[47] O. Kharlampovich and A. Myasnikov, Irreducible affine varieties over a free group. II. Sys-
tems in triangular quasi-quadratic form and description of residually free groups, J. Algebra
200 (1998), no. 2, 517–570, DOI 10.1006/jabr.1997.7184. MR1610664 (2000b:20032b)
[48] O. Kharlampovich and A. Myasnikov, Implicit function theorem over free groups, J. Algebra
290 (2005), no. 1, 1–203, DOI 10.1016/j.jalgebra.2005.04.001. MR2154989 (2007b:20047)
[49] O. Kharlampovich and A. G. Myasnikov, Effective JSJ decompositions, Groups, languages,
algorithms, Contemp. Math., vol. 378, Amer. Math. Soc., Providence, RI, 2005, pp. 87–212,
DOI 10.1090/conm/378/07012. MR2159316 (2006m:20045)
[50] O. Kharlampovich and A. Myasnikov, Elementary theory of free non-abelian groups, J.
Algebra 302 (2006), no. 2, 451–552, DOI 10.1016/j.jalgebra.2006.03.033. MR2293770
(2008e:20033)
[51] O. Kharlampovich and A. Myasnikov, Hyperbolic groups and free constructions, Trans. Amer.
Math. Soc. 350 (1998), no. 2, 571–613, DOI 10.1090/S0002-9947-98-01773-5. MR1390041
(98d:20041)
[52] O. Kharlamapovich and A. Myasnikov, Algebraic Geometry over Free Groups, to appear.
[53] O. Kharlampovich and A. Myasnikov, Algebraic geometry over free groups: lifting solu-
tions into generic points, Groups, languages, algorithms, Contemp. Math., vol. 378, Amer.
Math. Soc., Providence, RI, 2005, pp. 213–318, DOI 10.1090/conm/378/07013. MR2159317
(2006f:20026)
[54] O. Kharlampovich and A. Myasnikov, Implicit function theorem over free groups and genus
problem, Knots, braids, and mapping class groups, AMS/IP Stud. Adv. Math., vol. 24, Amer.
Math. Soc., Providence, RI, 2001, pp. 77–83. MR1873109 (2002j:20048)
[55] O. Kharlamapovich, A. Myasnikov, V. Remeslennikov and D. Serbin, Subgroups of fully resid-
ually free groups: algorithmic problems, Cont. Math. 360.
[56] S. Kim and S. Oum, Hyperbolic Surface Subgroups of One-Ended Doubles of Free Groups,
preprint.
[57] D. Lee, On certain C-test words for free groups, J. Algebra 247 (2002), no. 2, 509–540, DOI
10.1006/jabr.2001.9001. MR1877863 (2002m:20043)
[58] S. Lipschutz, The conjugacy problem and cyclic amalgamations, Bull. Amer. Math. Soc. 81
(1975), 114–116. MR0379675 (52 #580)
[59] R. C. Lyndon, The equation a2 b2 = c2 in free groups, Michigan Math. J 6 (1959), 89–95.
MR0103218 (21 #1999)
[60] R. C. Lyndon and P. E. Schupp, Combinatorial group theory, Springer-Verlag, Berlin-New
York, 1977. MR0577064 (58 #28182)
[61] A. Myasnikov and V. Remeslennikov, Length functions on free exponential groups, Proc.
Intern. Conference in Analysis and Geometry, Omsk, 1995, 59-61.
[62] D. I. Moldavanskiı̆, Certain subgroups of groups with one defining relation (Russian), Sibirsk.
Mat. Ž. 8 (1967), 1370–1384. MR0220810 (36 #3862)
[63] J. C. O’Neill and E. C. Turner, Test elements and the retract theorem in hyperbolic groups,
New York J. Math. 6 (2000), 107–117. MR1772562 (2001f:20088)
[64] A. Yu. Olshanskiı̆, On residualing homomorphisms and G-subgroups of hyperbolic groups,
Internat. J. Algebra Comput. 3 (1993), no. 4, 365–409, DOI 10.1142/S0218196793000251.
MR1250244 (94i:20069)
[65] N. Peczynski and W. Reiwer, On cancellations in HNN-groups, Math. Z. 158 (1978), no. 1,
79–86. MR0470085 (57 #9852)
[66] V. N. Remeslennikov, ∃-free groups (Russian), Sibirsk. Mat. Zh. 30 (1989), no. 6, 193–197,
DOI 10.1007/BF00970922; English transl., Siberian Math. J. 30 (1989), no. 6, 998–1001
(1990). MR1043446 (91f:03077)
[67] E. Rips and Z. Sela, Cyclic splittings of finitely presented groups and the canonical JSJ de-
composition, Ann. of Math. (2) 146 (1997), no. 1, 53–109, DOI 10.2307/2951832. MR1469317
(98m:20044)
[68] G. Rosenberger, On one-relator groups that are free products of two free groups with cyclic
amalgamation, Groups—St. Andrews 1981 (St. Andrews, 1981), London Math. Soc. Lec-
ture Note Ser., vol. 71, Cambridge Univ. Press, Cambridge-New York, 1982, pp. 328–344.
MR679174 (84i:20030)
[69] G. Rosenberger, The isomorphism problem for cyclically pinched one-relator groups, J. Pure
Appl. Algebra 95 (1994), no. 1, 75–86, DOI 10.1016/0022-4049(94)90119-8. MR1289120
(95g:20040)
[70] G. Rosenberger, Zum Isomorphieproblem für Gruppen mit einer definierenden Relation
(German), Illinois J. Math. 20 (1976), no. 4, 614–621. MR0442097 (56 #485)
[71] Z. Sela, The isomorphism problem for hyperbolic groups. I, Ann. of Math. (2) 141 (1995),
no. 2, 217–283, DOI 10.2307/2118520. MR1324134 (96b:20049)
[72] Z. Sela, Diophantine geometry over groups. I. Makanin-Razborov diagrams, Publ. Math.
Inst. Hautes Études Sci. 93 (2001), 31–105, DOI 10.1007/s10240-001-8188-y. MR1863735
(2002h:20061)
[73] Z. Sela, Diophantine geometry over groups. II. Completions, closures and formal solutions,
Israel J. Math. 134 (2003), 173–254, DOI 10.1007/BF02787407. MR1972179 (2004g:20061)
[74] Z. Sela, Diophantine geometry over groups. III. Rigid and solid solutions, Israel J. Math.
147 (2005), 1–73, DOI 10.1007/BF02785359. MR2166355 (2006j:20060)
[75] Z. Sela, Diophantine geometry over groups. IV. An iterative procedure for validation of
a sentence, Israel J. Math. 143 (2004), 1–130, DOI 10.1007/BF02803494. MR2106978
(2006j:20059)
[76] Z. Sela, Diophantine geometry over groups. V1 . Quantifier elimination. I, Israel J. Math.
150 (2005), 1–197, DOI 10.1007/BF02785359. MR2249582 (2007k:20088)
[77] V. Shpilrain, Recognizing automorphisms of the free groups, Arch. Math. (Basel) 62 (1994),
no. 5, 385–392, DOI 10.1007/BF01196426. MR1274742 (95f:20061)
[78] W. Szmielew, Elementary properties of Abelian groups, Fund. Math. 41 (1955), 203–271.
MR0072131 (17,233e)
[79] E. C. Turner, Test words for automorphisms of free groups, Bull. London Math. Soc. 28
(1996), no. 3, 255–263, DOI 10.1112/blms/28.3.255. MR1374403 (96m:20039)
[80] B. A. F. Wehrfritz, Generalized free products of linear groups, Proc. London Math. Soc. (3)
27 (1973), 402–424. MR0367080 (51 #3322)
[81] H. Zieschang, Über Automorphismen ebener diskontinuierlicher Gruppen (German), Math.
Ann. 166 (1966), 148–167. MR0201521 (34 #1403)
Department of Mathematics, Fairfield University, Fairfield, Connecticut 06430
Department of Mathematics, United States Naval Academy, Annapolis, Maryland

21402
Fachbereich Mathematik, University of Hamburg, Bundestrasse 55, 20146 Hamburg,

Germany
Department of Statistics, Temple University, Philadelphia, Pennsylvania 19122

Volume 633, 2015
An application of a localized version

of an axiom of Ian Chiswell
Anthony M. Gaglione, Seymour Lipschutz, and Dennis Spellman

Abstract. Generalizing results of Rimlinger, Hoare showed that certain pre-
groups admit Lyndon length functions on their universal groups from which
an asssociated graph of groups follows. Chiswell introduced axiom (P6) which
characterizes precisely those pregroups admitting such length functions on
their universal groups. The prototypical example of a (P6)-pregroup is the
subset G1 ∪ G2 of the amalgamated free product G1 ∗A G2 where A is proper
in each of G1 and G2 . If, for example, x ∈ G1 \A and y ∈ G2 \A, then xy is not
defined in G1 ∪G2 . Note that, in the above event, both xa and a−1 y are defined
if and only if a ∈ A in which case au and ua are defined for all u ∈ G1 ∪ G2 .
Explicity, (P6) asserts that if xy is not defined but each of xa and a−1 y is
defined, then au and ua are defined for all u, In some sense the next simplest
scenario is the iterated amalagamated free product G1 ∗A1,2 G2 ∗A2,3 G3 where
Ai,j is proper in Gi and Gj and A1,2 ∩ A2,3 is proper in each of A1,2 and A2,3 .
Although G1 ∪ G2 ∪ G3 is not a pregroup it is a pree in the sense of Lipschutz.
We introduce a localized version of Chiswell’s axiom and use it to characterize
the “hub” G2 in this and similar scenarios. We believe local (P6) is interesting
in its own right.
1. Introduction
Stallings, to some extent anticipated by Baer, introduced the concept of pre-
group to capture presentations presenting groups admitting reduced forms on the
beknighted generators. We begin with a structure P consisting of a set P admitting
a partial binary operation m : D → P where D ⊆ P × P . We write xy for m(x, y)
and say that xy is defined if (x, y) ∈ D.
Definition 1.1. In the above context and with the above notation a pree
P shall be a set P provided with a partial operation m : D → P , an involution
i : P → P , i(x) = x−1 , and a distinguished element 1 ∈ P subject to the following
four axioms:
(P1) For all x ∈ P , both 1x and x1 are defined and each is equal to x.
(P2) For all x ∈ P , both xx−1 and x−1 x are defined and each is equal to 1.
(P3) y −1 x−1 is defined whenever xy is defined; moreover, in that event, y −1 x−1
= (xy)−1 .
2010 Mathematics Subject Classification. Primary 20E06.

Key words and phrases. Pree, pregroup, tree pree, hub, hubbable.
2015
59
60 ANTHONY M. GAGLIONE, SEYMOUR LIPSCHUTZ, AND DENNIS SPELLMAN
(P4) If xy and yz are both defined, then (xy)z is defined if and only if x(yz) is
defined; moreover, in that event, they are equal and we say xyz is defined.
A pregroup will be a pree satisfying an additional axiom (P5) which we shall
presently make explicit after a pair of remarks and a definition.
Remark 1.2. Our terminology differs from that of Rimlinger [5] who used the
term “pree” to mean merely a set provided with a partial binary operation.
Remark 1.3. We are being faithful to Stallings’ original treatment in our
choice of axioms; however, it was shown that axiom (P3) in our definition of pree
is redundant since it follows from (P1), (P2) and (P4). (See e.g. Hoare [2].)
Definition 1.4. Let P be a pree. The universal group U (P) is the group
presented with generators P and defining relators xy = m(x, y) as (x, y) varies over
D. We say that P is embeddable provided the function P → U (P), x → x, for
all x ∈ P , is injective.
Definition 1.5. The pree P is a pregroup provided it satisfies the following
additional axiom:
(P5) If xy, yz and zw are all defined, then either xyz or yzw is defined.
Proposition 1 (Stallings [7]). Every pregroup is embeddable.
Now, following Serre [6], let (G, T ) be a tree of groups with vertex groups
{Gv : v ∈ vert(T )} and edge groups Ge = Gu ∩ Gv if e = {u, v} ∈ edge(T ). Let

P = Gv .
v∈vert(T )
We can make P into a pree P by taking

D= (Gv × Gv )
v∈vert(T )
and defining m on D such that the restriction of m on each Gv ×Gv is multiplication

in Gv . Such P are called tree prees. The universal group U (P) of a tree pree P
is just the corresponding tree product and so every tree pree is embeddable. In
general a tree pree need not be a pregroup. None the less every tree pree satisfies
the following axiom implied by (P5):
(K) If xy, yz, zw and (xy)(zw) are all defined, then either xyz or yzw is de-
fined.
Furthermore, if the diameter of the tree T is bounded by the positive integer
n, then the following axiom, implied by (P5), holds in the tree pree:
(Tn ) If x1 x2 , x2 x3 , ..., xn+2 xn+3 are all defined, then at least one of the triples
xj xj+1 xj+2 is defined for j = 1, 2, ..., n, n + 1.
Note that (P5) coincides with (T1 ) and that (Tn ) implies (Tk ) if k > n.
Proposition 2 (Kusher and Lipschutz [3] and [4]). Let P be a pree.

(1) If P satisfies (T 2 ), then P is embeddable.
(2) If P satisfies (T 3 ), and (K), then P is embeddable.
CHISWELL’S AXIOM 61
Remark 1.6. Part (1) of the above proposition was also done independently
by Hoare.
Let P be a pree. Then the base of P is

B(P) = {a ∈ P : Both ax and xa are defined ∀x ∈ P } .
One immediately convinces oneself that B(P) is a group.
In [1] Chiswell introduced the following axiom:
/ D but xa and a−1 y are both defined, then a ∈ B(P).
(P6) If (x, y) ∈
Proposition 3. Let P be a pree. If P satisfies (P6), then P is a pregroup.
Put another way, the proposition asserts that, in the presence of (P1), (P2)
and (P4), (P6) implies (P5). Before proving the proposition we observe that a tree
pree containing but a single edge satisfies (P6).
Proof of the Proposition. Assume xy, yz and zw are defined but xyz is
not defined. Since xyy −1 and yz are defined, y = (y −1 )−1 ∈ B(P) and thus yzw is
defined.
In [1] Chiswell introduces yet another axiom, namely:
(P7) ax is defined for all x ∈ P if and only if xa is defined for all x ∈ P .
He shows that (P7) is equivalent to each of the superficially weaker axioms:
(P7 ) If ax is defined for all x ∈ P , then xa is defined for all x ∈ P.
and
(P7 ) If xa is defined for all x ∈ P , then ax is defined for all x ∈ P .
Moreover, he proves that, in a pree, (P6) implies (P7). He gives an explicit
example of a pregroup which violates (P7) and thus also violates (P6). It follows
that the converse of our Proposition 3 is false. (Chiswell also gives an example of
a pregroup which satisfies (P7) but violates (P6).)
Definition 1.7. Let P = (P, DP , mP , iP , 1P ) and Q = (Q, DQ , mQ , iQ , 1Q ) be

prees. We shall say the Q is a (P6)-subpree of P provided the following six
conditions hold:
(1) Q satisfies (P6),
(2) Q ⊆ P,
(3) DQ = DP ∩ (Q × Q),
(4) m Q = m P |D Q ,
(5) iQ = iP |Q ,
(6) 1Q = 1P .
Definition 1.8. Let P be a pree. P satisfies local (P6) provided the following
two conditions hold:
(1) ∀(x, y) ∈ / D ∃ a unique (P6)-subpree M (x, y) maximal with respect to
containing {x, y}.
(2) ∀(x, y) ∈/ D if both xa and a−1 y are defined, then M (xa, a−1 y) = M (x, y).
(Note that, by (P1), (P2) and (P4), xy = [(xa)a−1 ]y is defined if and only
if (xa)[a−1 y] is defined.)
Proposition 4. Let P be a pree satisfying local (P6). If (x, y) ∈

/ D but xa and
a−1 y are both defined, then a ∈ M (x, y).
Proof. x ∈ M (x, y); hence, x−1 ∈ M (x, y). Moreover, xa ∈ M (xa, a−1 y) =
M (x, y). Thus, a = x−1 xa ∈ M (x, y).
Remark 1.9. It follows from Proposition 4 that B(P) is contained in every
M (x, y). If a ∈ B(P), then a−1 ∈ B(P) and both xa and a−1 y are defined ∀(x, y).
The following is an example (actually a family of examples) satisfying local
(P6) which is not a pregroup. We shall call it the standard example.
Example 1.10. Let the tree T contain exactly three vertices v1 , v2 and v3 with
v2 between v1 and v3 . Consider the tree of groups (G, T ) with vertex groups G1 , G2
and G3 corresponding to v1 , v2 and v3 respectively and edge groups A1,2 and A2,3
corresponding to edges {v1 , v2 } and {v2 , v3 } respectively. Assume A1,2 = G1 ∩ G2
is proper in each of G1 and G2 , A2,3 = G2 ∩ G3 is proper in each of G2 and G3
and, moreover, that G1 ∩ G3 = A1,2 ∩ A2,3 is proper in each of A1,2 and A2,3 . Then
the pree P corresponding to (G, T ) is not a pregroup.
If (x, y) ∈
/ D there are three possibilities. Namely:
(1) One of x and y lies in G1 \A1,2 and the other lies in G2 \A1,2 . In that event
{x, y} ⊆ M (x, y) = G1 ∪ G2 .
(2) One of x and y lies in G2 \A2,3 and the other lies in G3 \A2,3 . In that event
{x, y} ⊆ M (x, y) = G2 ∪ G3 .
(3) One of x and y lies in G1 \(A1,2 ∩A2,3 ) and the other lies in G3 \(A1,2 ∩A2,3 ).
In that event {x, y} ⊆ M (x, y) = G1 ∪ G3 .
We shall call G2 the hub of the standard example.
Proposition 5. Let P be a pree satisfying local (P6). Then
(1) P satisfies (K).
(2) P satisfies (P7).
Proof. (1) Suppose xy, yz, zw and (xy)(zw) are defined but xyz is not defined.
Consider M (x, yz). Since xy and y −1 yz are both defined, y ∈ M (x, yz). Moreover,
y ∈ B(M (x, yz)) as M (x, yz) satisfies (P6). From x ∈ M (x, yz) and y ∈ M (x, yz)
we get xy ∈ M (x, yz) as xy is defined. Since (xy)(zw) and (zw)−1 z are both
defined, zw ∈ M (x, yz). Since y ∈ B(M (x, yz)), yzw is defined.
(2) Suppose ax is defined for all x ∈ P . Assume to deduce a contradiction that
ya is not defined. Consider M (y, a), Now M (y, a) satisfies (P6) and thus also (P7).
Since ax is defined for all x ∈ M (y, a), we must have xa defined for all x ∈ M (y, a)
- contradicting the assumption that ya is not defined. The contradiction shows P
satisfies (P7).
Now let us reconsider Chiswell’s example of a pregroup which violates (P7).
Since every pregroup satisfies (K) we see that (modulo the pree axioms) (K) can-
not imply local (P6) for if it did, Chiswell’s example would satisfy (P7) by the
proposition.
Now let’s consider the standard example.
B(G1 ∪ G2 ) = A1,2 = A1,2 ∩ A2,3 = B(P),
B(G2 ∪ G3 ) = A2,3 = A1,2 ∩ A2,3 = B(P),
B(G1 ∪ G3 ) = A1,2 ∩ A2,3 = B(P).
CHISWELL’S AXIOM 63
G1 ∪ G3 is unique among the M (x, y) with the property that B(M (x, y)) = B(P).
If we take the intersection over the M (x, y) with B(M (x, y)) = B(P) we get the
hub G2 = (G1 ∪ G2 ) ∩ (G2 ∪ G3 ).
Definition 1.11. Let the pree P satisfy local (P6). Suppose among the M (x, y)
there is exactly one, call it M0 , such that B(M (x, y)) = B(P). Then P is hubbable
and its hub H(P) is
M (u, v)
M (u,v)=M0
provided there exist M (u, v) = M0 and is P otherwise.
Example 1.12. Let P be any (P6) pregroup that is not a group. In that event
P itself is the only M (x, y). So P is hubbable but H(P) = P is not a group.
Definition 1.13. Let the pree P be hubable. P is group hubbable provided

H(P) is a group.
Observe that in the standard example there are exactly three distinct M (x, y).
Lemma 1.14. Let P be a pree satisfying local P(6). The intersection of any
family of at least two distinct M (x, y) is a group.
Proof. Let I be such a intersection. It will suffice to show that xy is defined

for all (x, y) ∈ I × I. Suppose for some (x0 , y0 ) ∈ I × I we have that (x0 , y0 ) ∈
/ D.
Then there is a unique M (x0 , y0 ) containing {x0 , y0 }. But each of x0 and y0 lies in
every M (x, y) in the intersecting family - contradicting the uniqueness of M (x0 , y0 )
since the family contains at least two distinct members.
Theorem 1.15. Let the pree P be hubbable. If there are at least three distinct
M (x, y), then P is group hubbable.
Proof. Suppose P is a hubbable pree which contains at least three distinct
M (x, y). Let M0 be the unique M (x, y) with B(M (x, y)) = B(P) and let

H(P) = M (u, v)
M (u,v)=M0
be its hub. Then H(P) is a group since there are at least two distinct M (u, v) = M0 .

Remark 1.16. The “outer groups” can be captured as the M (u, v) ∩ M0 where
M (u, v) = M0 and M0 is the unique M (x, y) with B(M (x, y)) = B(P).
2. Questions
Question 1. Let P be a hubbable pree. If P contains at least two distinct

M (x, y) must it be group hubbable?
Question 2. Let P be a pree satisfying local (P6). Let us construct the
complete graph with vertices the distinct M (x, y). We introduce the vertex group
U (M (x, y)) at the vertex M (x, y) and for each edge connecting M (x1 , y1 ) with
M (x2 , y2 ) with M (x2 , y2 ) = M (x1 , y1 ) we introduce the edge group M (x1 , y1 ) ∩
M (x2 , y2 ) . (It is a group by Lemma 1.14.) Is U (P) isomorphic to the fundamental
group of the above graph of groups?
References
[1] I. M. Chiswell, Length functions and pregroups, Proc. Edinburgh Math. Soc. (2) 30 (1987),
no. 1, 57–67, DOI 10.1017/S001309150001796X. Groups—St. Andrews 1985. MR879430
(88e:20041)
[2] A. H. M. Hoare, Pregroups and length functions, Math. Proc. Cambridge Philos. Soc. 104
(1988), no. 1, 21–30, DOI 10.1017/S030500410006521X. MR938449 (89c:20048)
[3] Harvey Kushner, On Pre-Stars and Their Universal Groups, ProQuest LLC, Ann Arbor, MI,
1978. Thesis (Ph.D.)–Temple University. MR2627795
[4] Harvey Kushner and Seymour Lipschutz, On embeddable prees, J. Algebra 160 (1993), no. 1,
1–15, DOI 10.1006/jabr.1993.1174. MR1237074 (94i:20056)
[5] Frank Rimlinger, Pregroups and Bass-Serre theory, Mem. Amer. Math. Soc. 65 (1987), no. 361,
viii+73, DOI 10.1090/memo/0361. MR874086 (88i:20046)
[6] Jean-Pierre Serre, Trees, Springer-Verlag, Berlin-New York, 1980. Translated from the French
by John Stillwell. MR607504 (82c:20083)
[7] John Stallings, Group theory and three-dimensional manifolds, Yale University Press, New
Haven, Conn.-London, 1971. MR0415622 (54 #3705)
Department of Mathematics, U.S. Naval Academy, Annapolis, Maryland 21402

E-mail address: amg@usna.edu
URL: http://www.usna.edu
Department of Mathematics, Temple University, Philadelphia, Pennsylvania 19122

E-mail address: seymour@temple.edu

Volume 633, 2015
A note on Stallings’ pregroups
Anthony M. Gaglione, Seymour Lipschutz, and Dennis Spellman
Abstract. Stallings introduced the notion of a pregroup in order to capture

abstractly the structure of group amalgams. His definition depended on ax-
ioms satisfied by what is called a pree. Various authors including Hoare and
Rimlinger gave equivalent conditions for certain axioms. Kushner generalized
one of Stallings’ axioms. In the present paper, the authors continue along
these lines and give a condition that implies Kushner’s generalization.
1. Introduction
Among group presentations those which give standard forms of amalga-
mated free products or HNN extensions are special in that they yield normal forms.
Stallings (who was somewhat anticipated by Baer) defined the notion of pregroup
to treat such presentations uniformly. That he was successful may be found in a
theorem formulated in Rimlinger’s monograph [11] on pregroups and Bass-Serre
Theory. (Viz. Theorem 3 and the discussion which follows on pp. 2 and 3.) For
our purposes a pree shall be a set P provided with a distinguished element 1 ∈ P ,
a unary operation P → P, x → x−1 and a partial operation
m : D → P where D ⊆ P × P
subject to the first four of Stallings’ axioms, which he had denoted [P1] through
[P5]. (We make these explicit in the next section.) We would be remiss if we did not
point out here that our terminology differs from that of Rimlinger [11] for whom
a pree is a nonempty set provided with a partial binary operation. We should also
point out that it was shown that [P3] follows from [P1], [P2] and [P4] so may safely
be omitted from the axioms. (See e.g. [4]). One can give a categorical description
of the universal group G(P ) of a pree P ; however, for our purposes its description
on the generators P with the defining relations xy = m(x, y) for all (x, y) ∈ D will
suffice. We find it convenient to follow standard conventions abbreviating m(x, y)
as xy and replacing (x, y) ∈ D with the assertion, “xy is defined.” Then [P5] asserts
that if xy, yz and zw are all defined, then either xyz or yzw is defined.
Kushner and Lipschutz considered tree prees. Given a tree of groups we get a
pree P by taking P as the union of the vertex groups with xy defined provided x
and y lie in the same vertex group and m(x, y) being the product in that group.
2010 Mathematics Subject Classification. Primary 20E06.

Key words and phrases. Add, pree, pregroup.
2015
65
If there is a bound n on the diameter of the tree, then P satisfies the following
generalization of [P5]:
[Tn] If x1 x2 , x2 x3 , ..., xn+2 xn+3 are all defined then at least one triple xi xi+1 xi+2
is defined for i = 1, 2, ..., n, n + 1.
(Note that [T1]=[P5].)
Generalizing Stallings’ result that every pregroup embeds in its universal group,
Kushner [6] proved that every [T2]-pree is embeddable. Furthermore, Kushner and
Lipschutz [7] proved that every [T3]-pree subject to an additional axiom [K] (true
in every tree pree) is emebddable.
Conditions on so-called “heights” of elements facilitate proofs of structure re-
sults for their universal groups in terms of the Bass-Serre Theory. To capture the
notion of height one must use Stallings’ binary relation ≤. Here x ≤ y means
that for every z ∈ P , zx is defined whenever zy is defined. Then ≤ induces a
partial order on the equivalence classes where x ∼ y whenever x ≤ y and y ≤ x
hold simultaneously. The height of an element x (if it exists) is the length h of a
maximal chain
[1] = [x0 ] < [x1 ] < · · · < [xh ] = [x].
Rimlinger [11] proved structure results for the universal group of a pregroup under
the hypothesis that there be a uniform bound N on the heights of its elements.
Hoare [4] generalized Rimilinger’s results by showing it sufficed for ever element of
the pregroup to have finite height. In that same paper [4] in which Hoare relaxed
Rimlinger’s hypothesis, he proved that [P5] is equivalent to the following axiom
(which we call [GLS1]):
[GLS1] If xy −1 is defined, then x ≤ y or y ≤ x.

We introduce [GLS2]:
[GLS2] If xy −1 and y −1 z are defined, then at least one of x ≤ y, or y ≤ x, or
−1
y ≤ z −1 or z −1 ≤ y −1 holds.
We also introduce [GLS3]:
[GLS3] If x−1 y, y −1 z and z −1 w are all defined, then at least one of x ≤ y, or
y ≤ x, or y −1 ≤ z −1 or z −1 ≤ y −1 , or z ≤ w or w ≤ z holds
We show that [GLSn] implies [Tn] for n = 2 and n = 3. The question of
whether or not [Tn] implies [GLSn] for n = 3 or n = 3 remains open. We shall
repeat and expand more carefully on some of the definitions and examples above
in the main body of this paper.
2. Adds, Prees and Pregroups
Let P be a nonempty set with a partial operation m : D → P where D ⊆

P × P . This was called an “add” by Baer [1] who denoted m(p, q) by p + q in
contradistinction to our choice of the notation pq for m(p, q). If n is a positive
integer and X = (a1 , a2 , ..., an ) ∈ P n is an n-termed sequence in P we shall say
that X is defined if each pair a1 a2 , a2 a3 , ..., an−1 an is defined. By a triple in
X, we mean a subsequence (ai , ai+1 , ai+2 ).
PREGROUPS 67
Definition 2.1. The universal group G(P) of an add P is the group with
presentation
G(P ) = gp(P ; operation m).
That is, P is the set of generators for G(P ) and the defining relations are of
the form z = xy where m(x, y) = z.
Definition 2.2. An add P is said to be group-embeddable or simply em-

beddable if P can be embedded in its universal group G(P ).
An add P will be called a BS-pree or simply a pree if it satisfies the following

three axioms of Stallings:
[P1] (Identity) There exists 1 ∈ P such that for all a ∈ P , we have 1a and a1
are defined and 1a = a1 = a.
[P2] (Inverses) For each a ∈ P , there exists a−1 ∈ P such that aa−1 and a−1 a
are defined, and aa−1 = a−1 a = 1.
[P4] = [A] (Weak Associative Law) If ab and bc are defined, then (ab)c is defined
if and only if a(bc) is defined, in which case (ab)c = a(bc). (We then say the triple
abc is defined.)
Remark 2.3. Stallings also gave the axiom:

[P3] If ab is defined, then b−1 a−1 is defined and (ab)−1 = b−1 a−1 .
However, one can show that [P3] follows from [P1], [P2], and [P4].
It is not difficult to show:
Proposition 1. Inverses are unique in a pree P .
Proposition 2. If ab is defined in a pree P , then:
−1 −1
(i) (ab)b is defined and (ab)b = a,
(ii) a −1 (ab) is defined and a −1 (ab) = b.
We next give four classical examples of prees. The first three of which are
embeddable while the last one need not be.
Example 2.4. Let K and L be groups with isomorphic subgroups A, pictured

in Figure 1. Then the amalgam P = K ∪A L is a pree which is embeddable in
G(P ) = K ∗A L, the free product of K and L with A amalgamated.
A
K ——————— L
Figure 1
Example 2.5. Let K, H, L be groups. Suppose K and H have isomorphic

subgroups A, and suppose H and L have isomorphic subgroups B, pictured in
Figure 2. Then the amalgam P = K ∪A H ∪B L is a pree which is embeddable
in G(P ) = K ∗A H ∗B L the free product of K, H, L with subgroups A and B
amalgamated.
A B
K —————— H —————— L
Figure 2
Example 2.6. Let T = (Ki ; Ars ) be a tree graph of groups with vertex groups
Ki , and with edge
groups Ars . (Here Ars is a subgroup of vertex groups Kr and
Ks .) Let P = (Ki ; Ars ), the amalgam of the groups in T . Then P is a pree
i
which is embeddable in G(P ) = ∗(Ki ; Ars ), the tree product of the vertex groups
Ki with the Ars amalgamated.
Example 2.7. Let G = (Ki ; Ars ) be a graph of groups with vertex groups Ki
and withedge groups Ars . Again Ars is a subgroup of vertex groups Kr and Ks .
LetP = (Ki ; Ars ). Then P is a pree but , when the graph is not a tree, P need
i
not be embeddable in G(P ) = ∗(Ki ; Ars ), the amalgamated product of the graph
of groups Ki with the Ars . In fact, there are examples where G(P ) = {1}.
Stallings (1971) invented the name “‘pregroup” for a pree satisfying the follow-
ing additional axiom:
[P5] = [T1] If ab, bc, and cd are defined, then abc or bcd is defined.
Notation: If X is a set of axioms, then an X-pree will be a pree which also
satisfies the axioms in X.
With that convention we have, for example, that a pregroup is a T1-pree.
Theorem 2.8 (Stallings [12]). A pregroup P is embedded in G(P ).
What this means is that the pree morphism P → G(P ), x → x for all x ∈ P ,
is injective. We give an idea of his argument below. The details may be found in
[12].
We start out with sequences (x1 , x2 , ..., xn ) ∈ P n (where n is not fixed) and
think of such as words x1 x2 · · · xn on the generators P in G(P ). (Note that we do
not need negative exponents as x−1 ∈ P whenever x ∈ P .) If the product of any
two consecutive letters xi xi+1 is defined in P , then we may shorten the length of a
word representing the same element. So we further restrict ourselves to reduced
sequences (x1 , x2 , ..., xn ), namely: those for which no two consecutive xi xi+1 is
defined in Ṗ . Now
(x1 a1 )(a−1 −1 −1
1 x2 a2 ) · · · (an−2 xn−1 an−1 )(an−1 xn )
represents the same element of G(P ) as x1 x2 · · · xn. We therefore say that the
result of the interleaving of X = (x1 , x2 , ..., xn ) ∈ P n by A = (a1 , a2 , ..., an−1 ) ∈
P n−1 (where X and A are such that ai−1 xi a−1 i is defined i = 1, , 2, ..., n and here
a0 = an = 1), namely:
(x1 a1 , a−1 −1 −1
1 x2 a2 , · · · , an−2 xn−1 an−1 , an−1 xn )
is equivalent to (x1 , x2 , ..., xn ). We get a natural group structure on the equivalence

classes of reduced sequences modulo interleavings. Stallings uses a permutation
representation to show that this gives a faithful description of G(P ) into which P
embeds.
Reinhold Baer [1] also considered the embedding of prees. In particular, the
following appears in his paper where “exists” means “defined”:
PREGROUPS 69
Postulate XI: (Consists of three parts)

(a) If ab, bc, cd exist, then a(bc) or (bc)d exist.
(b) If bc, cd and a(bc) exist, then ab or (bc)d exist.
(c) If ab, bc and (bc)d exist, then a(bc) or cd exist.
Baer then states:
“In certain instances it is possible to deduce properties (b), (c) from (a); but
whether or not this is true in general, the author does not know.”
The following theorem (Lipschutz and Shi, [10]) answers Baer’s question:
Theorem 2.9. The following conditions on a pree P are equivalent.
(i) [P5] = [T1]: If ab, bc, cd are defined, then a(bc) or (bc)d is defined.
(ii) [A1]: If ab, (ab)c, ((ab)c)d are defined then bc or cd is defined.
(iii) [A2]: If cd, b(cd), a(b(cd)) are defined, then ab or bc is defined.
(iv) [A3]: If bc, cd, a(bc) are defined, then ab or (bc)d is defined.
(v) [A4]: If ab, bc, (bc)d are defined, then a(bc) or cd is defined.
Note [P5] = [T1] is Baer’s (a), [A3] is Baer’s (b) and [A4] is Baer’s (c).
Corollary 1. Let P be a pree which satisfies one of the axioms in Theorem

2.9 Then P is embeddable in its universal group G(P ).
The following transitive order relation on a pregroup P is due to Stallings.
Definition 2.10. Let L(x) = {a| ax is def ined}. Define x ≤ y if L(y) ⊆ L(x),
and define x < y if L(y) ⊂ L(x) but L(y) = L(x). We write [x] = [y] when
L(x) = L(y).
Example 2.11. Consider G = K ∗A L in Example 2.4. If x ∈ K\A, then

L(x) = K but if a ∈ A, then L(a) = K ∪ L = P .
The following theorem, due to Rimlinger [11] and Hoare [4], also gives equiva-
lent conditions to Stallings’ axiom [P5].
Theorem 2.12. The following conditions on a pree P are equivalent.

(i) (Axiom [P5] = [T1].) If wx , xy and yz are defined, then wxy or xyz is
defined.
(ii) If x −1 a and a −1 y are defined but x −1 y is not defined, then a < x and
a < y.
(iii) If x−1 y.is defined, then x ≤ y or y ≤ x.
3. Kushner’s Generalization of a Pregroup. T2-prees

Consider again G = K ∗A L in Example 2.4. Then P = K ∪ L is a pregroup
since [P5] = [T1] does hold in P . However, consider G = K ∗A H ∗B L in Example
2.5. Then P = K ∪ H ∪ L is not necessarily a pregroup since [P5] = [T1] need
not hold in P . For example, let x ∈ K\A, y ∈ L\B, a ∈ A, b ∈ B, as pictured in
Figure 3. Then xa ∈ K, ab ∈ H and by ∈ L are defined, but xab and aby need not
be defined (e.g., if also a ∈
/ B and b ∈
/ A).
x → K —————— H —————— L y
A B
↑ ↑
a b
Figure 3
On the other hand, if G = K ∗A H ∗B L, then P = K ∪ H ∪ L does satisfy the
axiom:
[T2] If ab, bc, cd, de are defined, then abc, bcd, or cde is defined.
That is, if X = (a, b, c, d, e) is defined, then a triple in X is defined.
Theorem 3.1 (Kushner [6]). Let P be a T2-pree. Then P is embeddable in
G(P ).
Independently, Hoare [5] also proved the above theorem.
The following theorem generalizes Bair’s question for the axiom [T2].
Theorem 3.2 (Gaglione, Lipschutz, Spellman, [2]). The following are equiva-
lent in a pree P where a, b, c, d, e are elements in P .
(i) [T2] If ab, bc, cd , de are defined, then a(bc), b(cd ), or c(de) is defined.
(ii) [B1] If bc, cd , a(bc), (cd )e are defined, then ab, (bc)d , or de is defined.
(iii) [B2] If ab, (ab)c, de, c(de) are defined, then bc, cd ,or (ab)c(de) is defined.
The following question was asked in [2]. Find analogous conditions using
Stallings inequality which is equivalent to [T2]. We partially answer this question
in this paper.
4. Axiom [GLS2]
Consider the following axiom:
[GLS2] If x−1 y and yz −1 are defined then either x ≤ y or y ≤ x or y −1 ≤ z −1
or z ≤ y −1 .
−1
Theorem 4.1. Axiom [GLS2] implies [T2].

Proof. Given a, b, c, d, e with ab, bc, cd, de defined. Apply [GLS2] with x =
b−1 , y = c and z = d−1 . We consider four cases.
(1) Suppose x ≤ y,that is, b−1 ≤ c. Then L(c) ⊆ L(b−1 ). Since (cd)−1 =
−1 −1
d c is defined and d−1 (c−1 c) = (d−1 c−1 )c is defined, we have (cd)−1 ∈ L(c) ⊆
L(b ). Thus (cd)−1 b−1 is defined and so ((cd)−1 b−1 )−1 = b(cd) is defined.
−1
(2) Suppose y ≤ x, that is, c ≤ b−1 . Then L(b−1 ) ⊆ L(c). Since a(bb−1 ) =
(ab)b−1 is defined, ab ∈ L(b−1 ) ⊆ L(c). Thus (ab)c is defined.
(3) Suppose y −1 ≤ z −1 , that is, c−1 ≤ d. Then L(d) ⊆ L(c−1 ). Since
e (d d) = (e−1 d−1 )d = (de)−1 d is defined, (de)−1 ∈ L(d) ⊆ L(c−1 ). Thus
−1 −1
(de)−1 c−1 is defined. Then ((de)−1 c−1 )−1 = c(de) is defined.

(4) Suppose z −1 ≤ y −1 , that is, d ≤ c−1 . Then L(c−1 ) ⊆ L(d). Since b(cc−1 ) =
(bc)c−1 is defined, bc ∈ L(c−1 ) ⊆ L(d). Thus (bc)d is defined (which is the same
conclusion as the conclusion in (1)).
Thus axiom [GLS2] implies [T2].
PREGROUPS 71
Corollary 2. Let P be a GLS2-pree. Then P is embeddable in G(P ).
5. Generalizations
Axiom [T2] can be generalized to the following axioms:

[T3] If X = (a, b, c, d, e, f ) is defined, then a triple in X is defined.
[Tn] If X = (a1 , a2 , ..., an+2 , an+3 ) is defined (i.e. each ai ai+1 is defined), then
a triple in X is defined.
Also, consider Kushner’s axiom:
[K] If X = (a, b, c, d) is defined, that is, if ab, bc, cd are defined, and if (ab)(bc)
is defined, then abc or bcd is defined.
Remark 5.1. Kushner and Lipschutz [8] proved that a T3K-pree is embed-
dable, and Lipschutz [9] proved that a TnK-pree is embeddble for n > 3. The
questions of whether a T3-pree (without K) is embeddable and whether a Tn-pree
(without K) is embeddable are still open questions.
Consider the following axiom

[GLS3] If x−1 y, yz −1 and z −1 w are defined then either x ≤ y or y ≤ x or
y ≤ z −1 or z −1 ≤ y −1 or z ≤ w or w ≤ z.
−1
Theorem 5.2. Axiom [GLS3] implies [T3].
Proof. Given a, b, c, d, e, f in P with ab, bc, cd, de, ef defined. Recall that
[T3] states that abc, bcd, cde, or def is defined. Apply [GLS3] with x = b−1 , y =
c, z = d−1 , w = e. We consider six cases.
(1) Suppose x ≤ y that is, b−1 ≤ c. Then L(c) ⊆ L(b−1 ). Since(cd)−1 = d−1 c−1
and d−1 (c−1 c) = (d−1 c−1 )c is defined, we have (cd)−1 ∈ L(c) ⊆ L(b−1 ). Thus
(cd)−1 b−1 is defined and so ((cd)−1 b−1 )−1 = b(cd) is defined.
(2) Suppose y ≤ x, that is, c ≤ b−1 . Then L(b−1 ) ⊆ L(c). Since a(bb−1 ) =
(ab)b−1 is defined, ab ∈ L(b−1 ) ⊆ L(c). Thus (ab)c is defined.
(3) Suppose y −1 ≤ z −1 , that is, c−1 ≤ d. Then L(d) ⊆ L(c−1 ). Since
e (d d) = (e−1 d−1 )d = (de)−1 d is defined, (de)−1 ∈ L(d) ⊆ L(c−1 ). Thus
−1 −1
(de)−1 c−1 is defined. Then ((de)−1 c−1 )−1 = c(de) is defined.

(4) Suppose z −1 ≤ y −1 , that is, d ≤ c−1 . Then L(c−1 ) ⊆ L(d). Since b(cc−1 ) =
(bc)c−1 is defined, bc ∈ L(c−1 ⊆)L(d). Thus (bc)d is defined (which is the same
(5) Suppose z ≤ w, that is, d−1 ≤ e. Then L(e) ⊆ L(d−1 ). Since ef is defined,
we have f −1 e−1 and (f −1 e−1 )e are defined. Thus f −1 e−1 ∈ L(e) ⊆ L(d−1 ).
Therefore f −1 e−1 (d−1 ) is defined. Accordingly, def is defined.
(6) Suppose w ≤ z, that is, e ≤ d−1 . Then L(d−1 ) ⊆ L(e). Since c(dd−1 ) =
(cd)d−1 is defined, cd ∈ L(d−1 ) ⊆ L(e). Thus (cd)e is defined (which is the same
Thus axiom [GLS3] implies axiom [T3].
Remark 5.3. Remark 5.1 tells us that we cannot conclude that a GLS3-pree
is embeddable, but that a GLS3K-pree is embeddable.
Lastly, we state a generalization of axiom [K] = [K1]. Specifically:

[K2] If X = (a, b, c, d, e) is defined, and if (ab)(cd) or (bc)(de) is defined, then
abc, bcd, or cde is defined.
[K3] If X = (a, b, c, d, e, f ) is defined, and if (ab)(cd) or (bc)(de) or (cd)(ef ) is
defined, then abc, bcd, cde, or def is defined.
More generally:
[Kn] If X = (a1 , a2 , .., an−1 , an−2 , an−3 ) is defined, and if one of (ai ai+1 )
(ai+2 ai+3 ) is defined, then a triple in X is defined.
Conjecture 1. Let P be a T3K3-pree or a GLS3K3-pree. Then P is embed-
dable in G(P ).
Conjecture 2. Let P be a TnKn-pree or a GLSnKn-pree. Then P is embed-
dable in G(P ).
Remark 5.4. Although we did not explicitly state [GLSn] for n > 3 its defini-
tion is clear.
References
[1] Reinhold Baer, Free sums of groups and their generalizations. II, Amer. J. Math. 72 (1950),
625–646. MR0038974 (12,478a)
[2] Anthony M. Gaglione, Seymour Lipschutz, and Dennis Spellman, Note on a question of
Reinhold Baer on pregroups II, Publ. Inst. Math. (Beograd) (N.S.) 92(106) (2012), 109–115,
DOI 10.2298/PIM1206109G. MR3051637
[3] Anthony M. Gaglione, Seymour Lipschutz, and Dennis Spellman, Survey of generalized pre-
groups and a question of Reinhold Baer, Algebra Discrete Math. 13 (2012), no. 2, 220–236.
MR3027508
[4] A. H. M. Hoare, Pregroups and length functions, Math. Proc. Cambridge Philos. Soc. 104
(1988), no. 1, 21–30, DOI 10.1017/S030500410006521X. MR938449 (89c:20048)
[5] A. H. M. Hoare, On generalizing Stallings’ pregroup, J. Algebra 145 (1992), no. 1, 113–119,
DOI 10.1016/0021-8693(92)90179-P. MR1144661 (92k:20057)
[6] H. Kushner, On prestars and their universal groups, Ph. D. Thesis, Temple University, 1987.
[7] Harvey Kushner and Seymour Lipschutz, A generalization of Stallings’ pregroup, J. Algebra
119 (1988), no. 1, 170–184, DOI 10.1016/0021-8693(88)90082-8. MR971352 (89m:20035)
[8] Harvey Kushner and Seymour Lipschutz, On embeddable prees, J. Algebra 160 (1993), no. 1,
1–15, DOI 10.1006/jabr.1993.1174. MR1237074 (94i:20056)
[9] Seymour Lipschutz, Generalizing the Baer-Stallings pregroup, Contemp. Math., vol. 169,
Amer. Math. Soc., Providence, RI, 1994, pp. 415–430, DOI 10.1090/conm/169/01672.
MR1292917 (95g:20038)
[10] Seymour Lipschutz and Wujie Shi, Note on a question of Reinhold Baer on pregroups, Publ.
Inst. Math. (Beograd) (N.S.) 68(82) (2000), 53–58. MR1826094
[11] Frank Rimlinger, Pregroups and Bass-Serre theory, Mem. Amer. Math. Soc. 65 (1987),
no. 361, viii+73, DOI 10.1090/memo/0361. MR874086 (88i:20046)
[12] John Stallings, Group theory and three-dimensional manifolds, Yale University Press, New
Haven, Conn.-London, 1971. MR0415622 (54 #3705)
Department of Mathematics, U.S. Naval Academy, Annapolis, Maryland 21402

E-mail address: amg@usna.edu
URL: http://www.usna.edu
Department of Mathematics, Temple University, Philadelphia, Pennsylvania 19122

E-mail address: seymour@temple.edu

Volume 633, 2015
A CCA secure cryptosystem using matrices over group rings
Delaram Kahrobaei, Charalambos Koupparis, and Vladimir Shpilrain
Abstract. We propose a cryptosystem based on matrices over group rings

and claim that it is secure against adaptive chosen ciphertext attack.
1. Cramer-Shoup cryptosystem
The Cramer-Shoup cryptosystem is a generalization of ElGamal’s protocol. It
is provably secure against adaptive chosen ciphertext attack (CCA). Moreover, the
proof of security relies only on a standard intractability assumption, namely, the
hardness of the Diffie-Hellman decision problem in the underlying group (see [2],
[3]), and a hash function H whose output can be interpreted as a number in Zq
(where q is a large prime number). An additional requirement is that it should
be hard to find collisions in H. In fact, with a fairly minor increase in cost and
complexity, one can eliminate H altogether.
1.1. Definition of provable security against adaptive chosen cipher-

text attack. A formal definition of security against active attacks evolved in a se-
quence of papers by Naor and Yung, Rackoff and Simon, Dolev, Dwork and Naor.
The notion is called chosen ciphertext security or, equivalently, non-malleability.
The intuitive thrust of this definition is that even if an adversary can get arbitrary
ciphertexts of his choice decrypted, he still gets no partial information about other
encrypted messages. For more information see [2], [3].
We define the following game, which is played by the adversary. First, we
run the enryption scheme’s key generation algorithm, with the necessary input
parameters. (In particular, one can input a binary string in {0, 1}n , which describes
the group G on which the algorithm is based.) The adversary is then allowed to
make arbitrary queries to the decryption oracle, decrypting ciphertexts which he
has chosen.
The adversary then chooses two messages, m0 and m1 , and submits these to
the encryption oracle. The encryption oracle chooses a random bit b ∈ {0, 1} and
encrypts mb . The adversary is then given the ciphertext, without knowledge of b.
2010 Mathematics Subject Classification. Primary 68.

Research of the first author was partially supported by a PSC-CUNY grant from the CUNY
research foundation, as well as the City Tech foundation.
Research of the third author was partially supported by the NSF grants DMS 0914778 and
CNS 1117675.
2015
73
74 D. KAHROBAEI, C. KOUPPARIS, AND V. SHPILRAIN
Upon receipt of the ciphertext from the encryption oracle, the adversary is
allowed to continue querying the decryption oracle. Of course the adversary is not
allowed to submit the output ciphertext of the encryption oracle.
Finally, at the end of the game, the adversary must output b ∈ {0, 1}, which
is the adversary’s best guess as to the value of b. Define the probability that b = b
to be 1/2 + (n), (n) is called the adversary’s advantage, and n ∼ |G|.
We say the cryptosystem is CCA-2 secure if the advantage of any polynomial-
time adversary is negligible. Note that a negligible function is a function that grows
slower than any inverse polynomial, n−c , for any particular constant c and large
enough n.
1.2. The Cramer-Shoup Scheme.
Secret Key: random x1 , x2 , y1 , y2 , z ∈ Zq
Public Key:
group G; g1 , g2 = 1 in G
c = g1 x1 g2 x2 , d = g1 y1 g2 y2
h = g1 z .
Encryption of m ∈ G: E(m) = (u1 , u2 , e, v), where
u1 = g1 r , u2 = g2 r , e = hr m, v = cr drα , where r ∈ Zq is random, and
α = H(u1 , u2 , e).
Decryption of (u1 , u2 , e, v):
If v = u1 x1 +αy1 u2 x2 +αy2 , where α = H(u1 , u2 , e),
then m = e/u1 z
else “reject”
1. Theorem: [2] The Cramer-Shoup cryptosystem is secure against adaptive
chosen ciphertext attack assuming that (1) the hash function H is chosen from a
universal one-way family, and (2) the Diffie-Hellman decision problem is hard in
the group G.
2. A CCA-2 secure cryptosystem using matrices over group rings

In [4], the authors proposed a public key exchange using matrices over group
rings. They offer a public key exchange protocol in the spirit of Diffie-Hellman, but
they use matrices over a group ring of a (rather small) symmetric group as the plat-
form and discuss security of this scheme by addressing the Decision Diffie-Hellman
(DDH) and Computational Diffie-Hellman (CDH) problems for that platform.
Here we propose to use a similar platform and show that a scheme similar to
the Cramer-Shoup scheme is CCA-2 secure. Our protocol is as follows:
Secret Key: random x1 , x2 , y1 , y2 , z ∈ Zn
Public Key:
3 × 3 non-identity matrices M1 , M2 ∈ M3×3 (Z7 [S5 ]) such that M1 is invertible and
M1 M2 = M2 M1
c = M1 x1 M2 x2 , d = M1 y1 M2 y2
h = M1 z .
Encryption of a message N ∈ M3×3 (Z7 [S5 ]): E(N ) = (u1 , u2 , e, v), where
u1 = M1 r , u2 = M2 r , e = hr N, v = cr drα , r ∈ Zn is random, and
α = H(u1 , u2 , e).
A CCA SECURE CRYPTOSYSTEM USING MATRICES OVER GROUP RINGS 75
Decryption of (u1 , u2 , e, v):

If v = u1 x1 +αy1 u2 x2 +αy2 , where α = H(u1 , u2 , e),
z −1
then N = (u1 ) e (Note that u1 is invertible since M1 is chosen to be invertible.)
else “reject”
Remarks: M1 must always be chosen to be an invertible matrix, whereas M2 is just
any matrix such that M1 M2 = M2 M1 . One must also decide what group Zn to
use, i.e., n must be specified.
3. Adaptive CCA security for matrices over group rings

We aim to show, by using Theorem 1, that if for invertible matrices over
M3×3 Z7 [S5 ] the DDH problem is hard, then the previously mentioned cyrptosystem
is secure against adaptive chosen ciphertext attack. More formally,
2. Theorem: The Cramer-Shoup cryptosystem using the semigroup G =
M3×3 Z7 [S5 ] is secure against adaptive chosen ciphertext attack assuming that (1)
the hash function H is chosen from a universal one-way family, and (2) the decision
Diffie-Hellman problem is hard in the group G.
Before beginning the proof of the theorem we need the following two experi-
mental facts.
(1) Given an invertible matrix M ∈ G = M3×3 Z7 [S5 ] and random integers
a, b and c ∈ N, it is not possible to distinguish between the distributions
generated by (M a , M b , M ab ) and (M a , M b , M c ).
(2) Given an invertible matrix M ∈ G = M3×3 Z7 [S5 ] and a random integer
a, it is not possible to extract information about a from M a and M . In
other words, the distributions generated by a random matrix N and M a
are indistinguishable.
We offer the following two experiments as evidence for the plausibility of
the above facts. For these tests we used invertible matrices over the group ring
M3×3 Z7 [S5 ]. For the first we chose a random invertible matrix M (see section
3.1.1) and random integers a, b and c ∈ N. We choose a and b in the interval
[1022 , 1027 ) and c in the interval [1044 , 1054 ) so that ab and c were roughly of the
same size. For each pair of resulting matrices M ab and M c we counted the frequency
of elements of S5 appearing in each entry.
Repeating this 500 times for randomly chosen a, b and c, we obtained a fre-
quency distribution of elements of the group ring in each entry of the two matrices.
From this we created the QQ-plots for each of the 9 matrix entries. QQ-plots are a
quick and easy way to test for identical distributions, in which case the plots should
be straight lines. As we can see from Figure 1, it appears that from the generated
distributions it is not possible to distinguish DH pairs from non-DH pairs.
For verification of the second fact, we conducted a similar experiment, except
in this case, for each of the 500 draws we varied all parameters N , M and a. We
again generated QQ-plots as shown in Figure 2, and these show that no information
about a is leaked from publishing M and M a .
We are now ready to prove Theorem 2. The proof will proceed in a similar
fashion as Cramer-Shoup’s original proof. We will begin by constructing an algo-
rithm D to attack the DDH assumption. This algorithm relies on a probabilistic
polynomial time adversary A attacking our scheme, which succeeds with proba-
bility p, PA (Success) = p. Denote by DH the set of valid Diffie-Hellman tuples
Figure 1. DDH results for M c vs. M ab
Figure 2. Results for M a vs. N
(M1 , M2 , M1r , M2r ), and by R the set of all random tuples (M1 , M2 , M3 , M4 ). Then
the algorithm is constructed as follows:
• D receives input (M1 , M2 , M3 , M4 ) from DH or R.
• Pick x1 , x2 , y1 , y2 , z ∈ Zn and a universal one-way hash function H.
• The adversary A receives the public key, PK, which is
(M1 , M2 , c = M1x1 M2x2 , d = M1y1 M2y2 , h = M1z , H).
• The adversary picks two messages m0 , m1 and publishes them.
• D picks b ∈ {0, 1} and passes to A

(M3 , M4 , M3z · mb , M3x1 +αx2 M4y1 +αy2 ),
where α = H(M3 , M4 , M3z · mb ).
• With this information A tries to determine b and returns its guess b .
• If b = b return “DH”, else “R”.
The proof is then verifying that this algorithm cannot attack the DDH prob-
lem. It is built from the following three claims.
Claim 1: |P(D = DH|DH) − P(D = DH|R)| < . This claim is trivially true
since D is a PPT algorithm and the DDH assumption holds as verified previously.
Claim 2: P(D = DH|DH) = PA (Success). If we are given a DDH tuple, then
all decryption queries succeed for A. Hence the output of A will match the choice
of b with PA (Success).
Claim 3: |P(D = DH|R) − 12 | < . Since P(D = DH) = P(A = b), the
proof of this claim relies on the proof of two pieces. We need to show that for all
decryption queries where u1 = M1r1 and u2 = M2r2 with r1 = r2 , the decryption
verification fails with non-negligible probability. In addition to this, we must also
show that assuming all invalid decryptions fail, the adversary A does not learn any
additional information about z.
We first start with the latter piece. If all invalid decryptions fail, then the only
additional information A receives is when valid decryptions are performed. Thus,
at the onset of the attack A only has information available that is given to him
from PK, namely h = M1z . If A submits a valid ciphertext (u1 , u2 , e , v ), where
r
u1 = M1r , then A obtains that hr = M1z . However, based on the results above,

if we denote M = M1z , then hr = M r and the distributions of any random matrix

N and M r generated by r are indistinguishable, hence nothing is revealed about
z.
Furthermore, from the encryption information passed to A, the only additional
information A has is M3z · mb , which leaves him with obtaining information from
M3z and M1z , i.e. solving a Diffie-Helmann problem, which we assumed was difficult
in our scheme setup.
We are now left with showing that decryption almost always fails for invalid ci-
phertexts. Suppose that the adversary submits an invalid ciphertext, (u1 , u2 , e , v )
= (u1 , u2 , e, v). Then we have the following cases:
Case 1: If (u1 , u2 , e) = (u1 , u2 , e ) and v = v , then the hash values α and α
will be the same, however decryption will certainly be rejected.
Case 2: If (u1 , u2 , e) = (u1 , u2 , e ) but a = a , then this means that A has
found a collision in H. But we assumed H was collision resistant, and since A runs
in polynomial time, this can only happen with negligible probability.
Case 3: If H(u1 , u2 , e) = H(u1 , u2 , e ), then we have the following system
of equations where we denote by log = logM1 and w = log(M2 ), and u1 = M1r1 ,
r r
u1 = M1 1 , u2 = M2r2 and u2 = M2 2 :
(1) log c =x1 + wx2 ,
(2) log d =y1 + wy2 ,
(3) log v =r1 x1 + wr2 x2 + αr1 y1 + αwr2 y2 ,
(4) log v =r1 x1 + wr2 x2 + α r1 y1 + α wr2 y2 .
These equations are linearly independent as can be verified by looking at

⎛ ⎞
1 w 0 0
⎜0 0 1 w ⎟
det ⎜ ⎟
⎝r1 wr2 αr1 αwr2 ⎠ = w (r2 − r1 )(r2 − r1 )(α − α ).
2
r1 wr2 α r1 α wr2

The above determinant is nonzero since we are considering bad decryptions and
hence
r1 = r1 , r2 = r2 , α = α .
Therefore, almost surely any bad decryption queries of this form will be rejected.
Thus we have shown from Claim 3 that the adversary A is unable to correctly
determine b given a random tuple, which we saw is equivalent to our algorithm not
being able to distinguish a random tuple from a DH tuple when given a random
tuple. This together with Claim 1 shows that our algorithm cannot distinguish
between tuples no matter what the input was. And finally, from Claim 2, we
get that the adversary is unable to attack our scheme with an adaptive chosen
ciphertext attack.
3.1. Parameters for the Cramer-Shoup-like scheme using matrices

over group rings. Here we address two problems relevant to key generation in
our scheme, namely, (1) how to sample invertible matrices and (2) how to sample
commuting matrices.
3.1.1. Invertible matrices. Sampling invertible matrices can be done using var-
ious techniques. The first method is to construct a matrix which is a product of
elementary matrices,
n
M= Ei ,
i=1
where Ei is any elementary matrix from M3×3 (Z7 [S5 ]). Elementary matrices can
be of one of the three types below. In the matrix Ti (u), the element u should be
invertible in Z7 [S5 ].
⎛1 ⎞
⎛ ⎞
1
⎜ .. ⎟
⎜ ⎟ ⎜ .. ⎟
⎜ . ⎟ ⎜ ⎟
⎜ ⎟ ⎜ . ⎟
⎜ 0 1 ⎟ ⎜ ⎟
⎜ ⎟ ⎜ 1 ⎟
⎜ .. ⎟ ⎜ ⎟
Ti,j =⎜ ⎟, Ti (u) = ⎜ u ⎟,
⎜ . ⎟ ⎜ ⎟
⎜ ⎟ ⎜ 1 ⎟
⎜ 1 0 ⎟ ⎜ ⎟
⎜ ⎟ ⎜ .. ⎟
⎜ .. ⎟ ⎝ . ⎠
⎝ . ⎠
1
1
⎛1 ⎞
⎜ .. ⎟
⎜ ⎟
⎜ . ⎟
⎜ ⎟
⎜ 1 ⎟
⎜ ⎟
⎜ .. ⎟
Ti,j (v) = ⎜ ⎟.
⎜ . ⎟
⎜ ⎟
⎜ v 1 ⎟
⎜ ⎟
⎜ .. ⎟
⎝ . ⎠
1
We can then easily compute M −1 as

n
−1
M −1 = En−i+1 .
i=1
The drawback of generating an invertible matrix this way is that we do not have a
good grasp of the randomness embedded in this process. In particular, how large
must n be to generate a truly random matrix? Given that there are 3 different types
of elementary matrices, does it matter in what order they are multiplied in and does
the number of elementary matrices of each form matter? These are questions that
have not been addressed and may influence the final invertible matrix generated in
unknown ways.
Here, instead of the previously mentioned method of sampling random matrices,
we propose an alternative solution. We start with an already “somewhat random”
matrix, for which it is easy to compute the inverse. An example of such a matrix
is a lower/upper triangular matrix, with invertible elements on the diagonal:
⎛ ⎞
u1 g1 g2
M = ⎝ 0 u2 g3 ⎠ .
0 0 u3
Constructing the inverse of this matrix involves solving a matrix equation,
M · M −1 = I
⎛ ⎞ ⎛ −1 ⎞ ⎛ ⎞
u1 g1 g2 u1 g4 g5 1 0 0
⇒⎝0 u2 g3 ⎠ · ⎝ 0 u−1
2 g6 ⎠ = ⎝0 1 0⎠
0 0 u3 0 0 u−1
3 0 0 1
⇒ g4 = −u−1 −1
1 g1 u2
g5 = u−1 −1 −1 −1 −1
1 g1 u2 g3 u3 − u1 g2 u3
g6 = −u−1 −1
2 g3 u3 .
We then propose to take a random product of such invertible upper and lower
triangular matrices. Since these matrices are more complex than elementary ma-
trices, it seems reasonable to assume that we arrive at a more uniform distribution
sooner than by simply using elementary matrices. In our experiments we used a
product of 20 random matrices, where each term of the product was chosen ran-
domly as either a random invertible upper or lower triangular matrix.
As mentioned previously, the benefits of this method are that inverses are easy
to compute and that the chosen matrix already has a large degree of randomness
built in. In particular, any element of Z7 [S5 ] can be used off the diagonal, and any
invertible elements of the group ring can be used on the diagonal. These of course
include elements such as nu ∈ Z7 [S5 ], where u ∈ S5 and n ∈ Z7 .
Finally, we note that the order of the group GL3 Z7 [S5 ] of invertible 3 × 3
matrices over Z7 [S5 ] is at least 10313 . Indeed, if we only count invertible upper
and lower triangular matrices that we described above, then we already have (7 ·
120)3 (7120 )3 ∼ 10313 matrices.
3.1.2. Commuting matrices. Now that we have sampled an invertible matrix
(M1 in our notation – see Section 2), we have to sample an arbitrary (i.e., not
necessarily invertible) matrix M2 that would commute with M1 .

Given a matrix M1 ∈ G, define M2 = ki=1 ai M1i , where ai ∈ Z7 are selected
randomly. Then clearly M1 M2 = M2 M1 . A reasonable choice for k is about 100 as
this would yield 7100 ∼ 1085 choices for M2 , which is a sufficiently large key space.
3.1.3. Other parameters. As mentioned in the introduction of the Cramer-
Shoup algorithm adapted to our group rings, we need to specify the value of n
for Zn . Based on experiments in our previous paper [4] we suggest n ∼ 10100 . This
seemed a reasonable choice of exponent since it both allowed quick computations
and ensured that the power a matrix was raised to could not be figured out by
brute force methods alone.
We also use a hash function H in our algorithm as did Cramer and Shoup. The
only requirement on H is that it is drawn from a family of universal one-way hash
functions. This is a less stringent requirement than to be collision resistant. The
latter implies that it is infeasible for an adversary to find two different inputs x and
y such that H(x) = H(y). A weaker notion of second preimage resistance implies
that upon choosing an input x, it is infeasible to find a different input y such that
H(x) = H(y).
It should be noted that in their paper Cramer and Shoup also give details of
their same algorithm without requiring the use of any hash functions. The modified
algorithm is only slightly more complicated but relies on the same principles.
References
[1] D. Boneh, The decision Diffie-Hellman problem, Algorithmic number theory (Portland, OR,
1998), Lecture Notes in Comput. Sci., vol. 1423, Springer, Berlin, 1998, pp. 48–63, DOI
10.1007/BFb0054851. MR1726060 (2000k:94024)
[2] R. Cramer and Victor Shoup, A practical public key cryptosystem provably secure against
adaptive chosen ciphertext attack, Advances in cryptology—CRYPTO ’98 (Santa Barbara,
CA, 1998), Lecture Notes in Comput. Sci., vol. 1462, Springer, Berlin, 1998, pp. 13–25, DOI
10.1007/BFb0055717. MR1670952 (99j:94041)
[3] V. Shoup, Why chosen ciphertext security matters, IBM Research Report RZ 3076, 1998.
[4] D. Kahrobaei, C. Koupparis, and V. Shpilrain, Public key exchange using matrices over group
rings, Groups Complex. Cryptol. 5 (2013), no. 1, 97–115. MR3065451
CUNY Graduate Center and City Tech, City University of New York
E-mail address: DKahrobaei@GC.Cuny.edu
CUNY Graduate Center, City University of New York

E-mail address: ckoupparis@GC.Cuny.edu
The City College of New York and CUNY Graduate Center

E-mail address: shpil@groups.sci.ccny.cuny.edu
Volume 633, 2015
The MOR cryptosystem and finite p-groups
Ayan Mahalanobis
Abstract. The ElGamal cryptosystem is the most widely used public-key

cryptosystem. It uses the discrete logarithm problem as the cryptographic
primitive. The MOR cryptosystem is a similar cryptosystem. It uses the
discrete logarithm problem in the automorphism group as the cryptographic
primitive. In this paper, we study the MOR cryptosystem for finite p-groups.
The study is complete for p -automorphisms. For p-automorphisms there are
some interesting open problems.
1. Introduction
This is a study of the MOR cryptosystem using finite p-groups. Similar studies
were done by this author [11, 12]. The MOR cryptosystem, that we are going to
describe in details shortly, works with the automorphism group of a group. As
a matter of fact, we do not even need a group. Any finitely presented structure
on which automorphisms can be defined will do. We can define the MOR cryp-
tosystem on that structure. However, a MOR cryptosystem might not be secure or
implementation-friendly. So this paper can be seen as a search for favorable groups
for the MOR cryptosystem in the class of finite p-groups.
Once we decide that we will look into the class of p-groups, an obvious question
surfaces. Are there p-groups on which the cryptosystem is secure? Once the answer
is yes, then is it any better than the existing one? So we have three questions in
front of us:
1: Are there favorable p-groups?
2: Is the cryptosystem secure1 on those groups?
3: Is the cryptosystem faster on those groups compared to a suitably defined
ElGamal cryptosystem?
To answer these questions, we had to divide the automorphisms in two different
classes. One, p-automorphisms and the other p -automorphisms. For p -automorph-
isms we show that there are secure MOR cryptosystems on a p-group. However,
they offer no advantage than working with matrices over the finite field Fp . So,
2010 Mathematics Subject Classification. Primary 94A60, 20D15.

Key words and phrases. MOR cryptosystem, finite p-groups, the discrete logarithm problem.
This research was supported by a NBHM research grant.
1 There are many different definitions of security, we use the basic one – find m, from the
automorphism φ and its power φm .
2015
81
82 AYAN MAHALANOBIS
after reading this paper, one might argue and rightfully so: instead of using p -
automorphisms and p-group, why not just use matrices of the right size?
The case for p-automorphisms is little complicated and we say upfront that
we have not been able to analyze it completely. The case of p-automorphisms
break down into two sub-cases and we were able to deal with one easily. The
other case is very interesting and we were able to shed some light into that with
an example. However, a detailed analysis is missing and we leave it as ongoing
research. The situation with p-automorphisms compared to p -automorphisms is in
many ways similar to the modular representation theory compared to the ordinary
representation theory. The later is much easier to deal with than the former.
2. Definitions and notations

Most of the definitions used in this paper are standard and in Gorenstein [3].
However, we define a few of them for the convenience of the reader. All groups in
this paper are finite. We use p for a prime and q for a prime-power.
Definition 2.1 (p -automorphisms and p-automorphisms). An automorphism
φ of a p-group G is a p-automorphism if its order is power of p and p -automorphism
if its order is coprime to p.
In general, it is not true that an automorphism is either a p-automorphism
or a p -automorphism. However, for the purpose of understanding the security of
a MOR cryptosystem, due to the Pohlig-Hellman algorithm [5, Section 2.9], an
automorphism is either a p-automorphism or a p -automorphism.
Definition 2.2 (Special p-group). Usually, a special p-group is defined to be
a p-group such that Z(G) = G = Φ(G) and is elementary-abelian. Here G , Z(G)
and Φ(G) are the commutator subgroup, the center and the Frattini subgroup
respectively. However, it is not very hard to show that the elemetary-abelian part
is redundant.
Definition 2.3 (Favorable p-group). A p-group G is called a favorable p-
group, if there is a non-identity p -automorphism φ of the group, such that, if the
automorphism fixes a proper subgroup H of G, it is the identity on H.
A good example of a favorable p-group is the elementary-abelian p-group, de-
noted by G. Any automorphism of that can be realized as a matrix. If the character-
istic polynomial of an automorphism φ is irreducible, then there are no φ-invariant
proper subgroups of G. So the above condition is true vacuously.
A curious reader might find the requirement “p -automorphism φ” unnecessary.
The reason for the restriction is, for p -automorphisms favorable p-groups is the
right notion to look at. If there is a subgroup that is fixed by φ, one can study the
discrete logarithm problem on the action of the automorphism on that subgroup,
unless the automorphism is the identity on that subgroup. We will see, in the case
of p -automorphisms, the discrete logarithm problem in the automorphism group
translates to the discrete logarithm problem in non-singular matrices. In the case
of p-automorphisms, it is not clear if the notion of favorable p-group is the best way
to go. We simply don’t have enough examples of secure MOR cryptosystem using
p-automorphisms of p-groups yet. So we refrain ourselves from defining favorable
p-groups for p-automorphisms.
THE MOR CRYPTOSYSTEM AND FINITE p-GROUPS 83
3. The MOR cryptosystem

In this section, we provide a somewhat detailed description of a small but im-
portant portion of public-key cryptography. We start with a cryptographic primitive
– the discrete logarithm problem. The standard reference for public-key cryptogra-
phy is Hoffstein et. al. [5].
Definition 3.1 (The discrete logarithm problem). Let G = g
be a finite
cyclic group of prime order. We are given g and g m for some m ∈ N. The discrete
logarithm problem is to find the smallest m.
The discrete logarithm problem is neither secure or insecure. It being secure or
insecure is a property of the presentation of the group. The property of being secure
or insecure is not a group theoretic property, it is not invariant under isomorphism.
The discrete logarithm problem is the easiest in prime subgroups of (Zn , +)
and is considered secure in prime subgroups of the multiplicative group of a finite
field F× q and is considered really secure in a prime order subgroup of the rational
points of an elliptic curve. The difference in security between finite fields and points
on elliptic curve is, there is no known sub-exponential attack against the elliptic
curves.
A concept related to the discrete logarithm problem is the Diffie-Hellman

problem. We have the same G as before, this problem is: given g, g m and

g m compute g m m . It is clear that if we know how to solve the discrete logarithm
problem, i.e., we can find m (or m ), we can then solve the Diffie-Hellman problem.
The reverse direction is not known.
The most popular and prolific public-key cryptosystem is the ElGamal cryp-
tosystem. It works in any cyclic subgroup of a group G. However, it might not be
secure in any group.
3.1. Description of the ElGamal cryptosystem.

Private Key: m, m ∈ N.
Public Key: g and g m .
Encryption.
a: To send a message (plaintext) a ∈ G Bob computes g r and g mr for a
random r ∈ N.
b: The ciphertext is (g r , g mr a).
Decryption.
a: Alice knows m, so if she receives the ciphertext (g r , g mr a), she computes
g mr from g r and then g −mr and then computes a from g mr a.
It is known that the security of the ElGamal cryptosystem is equivalent to the
Diffie-Hellman problem [5, Proposition 2.10]. A very similar idea is the MOR
cryptosystem.
3.2. Description of the MOR cryptosystem. Let G = g1 , g2 , . . . , gτ

,
τ ∈ N be a finite group and φ a non-trivial automorphism of G. Alice’s keys are as
follows:
Private Key: m, m ∈ N.
Public Key: {φ(gi )}τi=1 and {φm (gi )}τi=1 .
84 AYAN MAHALANOBIS
Encryption.
a: To send a message (plaintext) a ∈ G Bob computes φr and φmr for a
random r ∈ N.
b: The ciphertext is ({φr (gi )}τi=1 , φmr (a)).
Decryption.
a: Alice knows m, so if she receives the ciphertext (φr , φmr (a)), she computes
φmr from φr and then φ−mr and then computes a from φmr (a).
Alice knows the order of the automorphism φ, she can use the identity φt−1 = φ−1
whenever φt = 1 to compute φ−mr .
It is easy to see the following: if one can solve the Diffie-Hellman problem in
φ
, he can break the MOR cryptosystem. This follows from the fact that φr and
φm are public. If one can solve the Diffie-Hellman problem, one can compute φmr
and get the plaintext. The next theorem proves the converse.
Theorem 3.1. If there is an oracle that can decrypt arbitrary ciphertext for
the MOR cryptosystem, one can solve the Diffie-Hellman problem in φ
.
Proof. Assume that there is an oracle that can decrypt arbitrary MOR ci-

phertext. Now recall that a = φ−mr (φmr (a)). Now suppose we have φm and φm

and we wantto compute φm m . Then tell the oracle that φm is the public key

and φm , gi is the ciphertext. The oracle will return φ−m m (gi ) as the plaintext.

Once this game is played for i = 1, 2, . . . , τ . We know φ−m m (gi ) for i = 1, 2, . . . , τ

and hence φm m . Thus solving the Diffie-Hellman problem in φ
. •
In this paper we are primarily interested in exploring finite p-groups for the
purpose of building a secure MOR cryptosystem. As is well known, security and
computational efficiency goes hand in hand. So unless we explore the computational
complexity of the MOR cryptosystem, a security analysis is useless. So there are
two questions that we will explore in this paper:
a: Is it possible to build a secure MOR cryptosystem using finite p-groups?
b: Does this MOR cryptosystem has any advantage over existing cryptosys-
tems?
Before we answer these questions, we need to explain one aspect of the security
of the discrete logarithm problem. It is easy to see, using the Chinese remainder
theorem, that the discrete logarithm problem in any cyclic group can be reduced
to a discrete logarithm problem in its Sylow subgroups. Then a discrete logarithm
problem in the Sylow subgroup can be further reduced to the discrete logarithm
problem in a group of prime order [5, Section 2.9]. The end result is: the security of
the discrete logarithm problem in a group is the security of the discrete logarithm
problem in the largest prime-order subgroup in that group. In practice, the group
considered for an efficient and secure implementation of the discrete logarithm
problem is a group of prime order2 . From the above argument, it is clear that we
should only study automorphisms of prime order for the MOR cryptosystem.
One way to study automorphisms of a finite p-group for the MOR cryptosystem
is using linear methods in nilpotent groups [6, Chapter VIII]. That is our principal
2 The reader must have noticed that in the definition of the discrete logarithm problem we
used groups of prime order.

objective in this paper. The idea is to find a series of subgroups such that au-
tomorphisms act linearly either on the subgroups or the quotients. We will soon
assume, if a subgroup is fixed under an automorphism then it is the identity on
that subgroup. With this assumption, we only have to look at the action of an
automorphism on the sections of the series.
With these in mind, we look at the exponent-p central series of a finite p-group
G. The series is defined as follows:
G = G0 G 1 . . . G k = 1
where Gi+1 = [G, Gi ] Gpi .
This series is well known to have elementary-abelian
quotients and is used in many aspects of computations with finite p-groups [14].
There are two possible orders of an automorphism of a p-group for the MOR
cryptosystem:
i: The automorphism φ is of order p.
ii: The order of φ is a prime different from p, i.e., a p -automorphism.
This can again be subdivided into four different cases:
a: The automorphism is of order p and is identity on all the quotients of the
exponent-p central series.
b: The automorphism is of order p and is not identity on at least one section
of the exponent-p central series.
c: The automorphism is of order p and is not identity on at least one section
of the exponent-p central series.
d: The automorphism is of order p and is identity on all sections of the
exponent-p central series.
Recall that G1 is the Frattini subgroup Φ(G). A well known theorem of Burnside
says that:
Theorem 3.2 (Burnside). Let φ be an automorphism of a group G. If the
greatest common divisor, gcd (o(φ), |Φ(G)|) = 1 and φ induces the identity auto-
morphism on G/Φ(G), φ is the identity automorphism on G.
Proof. For a proof see [1, Theorem 1.15] or [3, Theorem 5.1.4]. •
This says, the case c above reduces to: the automorphism φ is of order p and
is not identity on G/Φ(G). In this case φ acts on G/Φ(G) linearly and the discrete
logarithm problem in φ deduces to the discrete logarithm problem in matrices over
Fp . The size of the matrix is the same as the cardinality of a set of minimal
generators of the p-group.
It is also well known, if d is the case then φ is the identity [3, Theorem 5.3.2].
So there is no point studying d.
So we have three cases to look at a, b and c above.
It is well known that usually, the exception being groups of prime order, p-
groups come with lots of subgroups and normal subgroups. The most difficult
issue that one faces in choosing a p-group and the automorphism φ for the MOR
cryptosystem is the presence of subgroups of the p-group which is fixed by φ. If
this happens, the discrete logarithm problem in the automorphism φ is reduced
to the discrete logarithm problem in the restriction of φ to that subgroup. This
reduction is most undesirable. On the other hand, working with non-abelian p-
groups this reduction is bound to happen. For example, the commutator and the
center are non-trivial characteristic subgroups. The way out of this situation is to
86 AYAN MAHALANOBIS
ensure, if φ fixes any subgroup then it is the identity on that subgroup. Once this
condition is imposed, we will see that favorable groups with p -automorphism are
reduced to either the elementary abelian p-group or the class of p-groups G with
G = Z(G) = Φ(G) and Φ(G) is elementary abelian. Here G is the commutator
subgroup, Z(G) is the center and Φ(G) is the Frattini subgroup of G. These two
class of groups together are known as special p-groups.
4. MOR cryptosystems on finite p-groups using p -automorphisms

In this section we look at the MOR cryptosystem over finite p-groups with
p -automorphisms. Our standard reference for group theory is Gorenstein [3] and
for linear algebra is Roman [15]. We start with a generalization of a celebrated
theorem from the odd-order paper.
Theorem 4.1. A solvable group G possesses a characteristic subgroup C with
the following properties:
• Subgroup C is nilpotent with nilpotency class less than or equal 2.
• Z(C) is a maximal characteristic abelian subgroup of G.
• CG (C) = Z(C).
• Every nontrivial p -automorphism of G induces a non-trivial automor-
phism on C.
For a proof see [1, Theorem 14.1]. The subgroup C is called a Thompson critical
subgroup. We will refer to it as a critical subgroup. There can be more than one
critical subgroup in a group. It is clear from the theorem above, in our search for
favorable p-groups, we should look at p-groups whose only critical subgroup is the
whole group. We will call those groups self-critical. Since a self-critical group is
of class at most 2, we should look at p-groups of class at most 2. Now if p is odd,
in a p-group of class 2, (xy)p = xp y p . This makes the subgroup Ω1 (G) of exponent
p. Since Ω1 (G) is characteristic the following corollary follows immediately.
Corollary 4.2. For an odd prime p, favorable p-groups are of class at most
2 with exponent p.
Before we go any further we need to state a well known theorem due to Hall
and Higman [4, Theorem C]. The proof is available in many standard textbooks [3,
Theorem 5.3.7], so we won’t reproduce it.
Theorem 4.3. Let G be a favorable p-group, then G/G is elementary abelian.
To summarize, favorable p-groups are of class at most 2 and G/G is elementary-
abelian. It follows that G ≤ Z(G). Then both G/G and G/Z(G) are elementary
abelian p-groups. We also have a p -automorphism φ, such that, if φ fixes a subgroup
of G, it is the identity on that subgroup. In particular, φ is the identity on G and
Z(G).
There are two different ways to look at this situation:
Ordinary representation theory. Let A = φ
be the subgroup generated
by φ. Since φ is a p -automorphism, the order of A is coprime to the order of the
group G. We have a coprime action of A on G. In particular, we have a linear
action of A on V = G/G . Since this action is coprime we have the celebrated
Maschke’s theorem [3, Theorem 3.3.1] at our disposal. The theorem states, if we
have an A-invariant proper subspace W ⊂ V , it has an A-invariant complement.
In other words there is an A-invariant subspace W of V such that V = W ⊕ W .
Linear algebra. Another way to look at the same situation is by linear al-
gebra. Let V = G/G . Clearly V is a finite dimensional vector-space over Fp .
Corresponding to a linear transformation φ of V , we can define scalar multipli-
cation such that V is a finitely generated module over the principal ideal domain
Fp [x] [15, Chapter 7]. We denote this module by Vφ . The reason we are interested
in this module Vφ is that the submodules of Vφ are the φ-invariant subspaces of V .
With this we have the full force of the theory of finitely generated modules over a
principal ideal domain at our disposition; especially the decomposition theorem.
The minimal polynomial of φ is a generator of the annihilator ideal of Vφ in
Fp [x]. We denote it by mφ and assume it to be monic. Let mφ = f1m1 (x)f2m2 (x) . . .−
fkmk (x) be the decomposition of mφ as product of irreducible monic polynomials.
One can write Vφ = V1 ⊕ V2 ⊕ . . . ⊕ Vk where a generator of the annihilator ideal
of each primary component Vi is fimi . Each Vi can either be cyclic or can be
broken down as direct sum of cyclic modules. This theory is very well-known and
successful, so we will omit the details and ask any interested reader to consult a
textbook in linear algebra – Roman [15] being one of them.
Lemma 4.4. Let φ be a non-identity p -automorphism on V , where V is a
finite-dimensional vector space over Fp ; such that, if φ fixes a subspace of V then
it is the identity on that subspace. The following is true:
a. The characteristic polynomial χφ of φ is irreducible.
b. The module Vφ is simple.
Proof. Recall that Vφ is a finitely generated module over a principal ideal
domain Fp [x]. Let mφ be the minimal polynomial of Vφ . Assume that mφ =
f1m1 (x)f2m2 (x) . . . fkmk (x), where each fi (x) is monic irreducible over Fp and each
mi is a non-negative integer. Define the set
Vi = {v ∈ Vφ : fimi (φ)v = 0} .
Then the fundamental theorem of finitely generated module over a principal ideal
domain says that Vφ = V1 ⊕ V2 ⊕ . . . ⊕ Vk . Now assume for a moment that k is
greater than 1. Then we have Vφ as direct sum of non-trivial submodules. Recall
that submodules of Vφ are the φ-invariant subspaces of V . Then we have that V
is a direct sum of two φ-invariant subspaces of V . So φ acts like identity on both
these subspaces and hence is the identity on V . So this subspace decomposition is
impossible, forcing k to be 1.
We have deduced that mφ = f (x)l where f (x) is monicirreducible and l is a pos-
itive integer. If l is greater than 1, take the subspace V = v ∈ Vφ : f l−1 (φ)v = 0 .
Also construct the subgroup A = φ
. Since gcd(|A|, p) = 1, from Maschke’s the-
orem the subspace V has a complement. This means that there is another A-
invariant subspace V such that V = V ⊕ V . Then using an argument similar to
the one in last paragraph, we show that l = 1 and the minimal polynomial mφ is
irreducible.
From the above discussion it follows clearly that the module Vφ is cyclic with
irreducible minimal polynomial. Since a cyclic module with irreducible minimal
polynomial is non-derogatory [15, Theorem 7.11], we have the characteristic poly-
nomial the same as the minimal polynomial.
The fact the module is simple, follows from the fact that the minimal polyno-
mial of any submodule will divide the minimal polynomial of the module and the
minimal polynomial of the module is irreducible. •
88 AYAN MAHALANOBIS
It is easy to prove a partial converse of the above lemma.

Lemma 4.5. Let φ be a linear transformation on the finite dimensional vector
space over Fq . If the characteristic polynomial χφ is irreducible, the only φ-invariant
subspaces of V are 0 and V .
Proof. We will consider Vφ as a module over Fq [x]. Since χφ is irreducible it
is also the minimal polynomial. Now if S is a submodule of Vφ , then its minimal
polynomial will divide χφ . Since χφ is irreducible, we have a proof. •
This lemma is the most useful lemma in this whole paper. This paper is in
search of favorable p-groups and the corresponding automorphism. One way, and
probably the easiest way, is to look at the characteristic polynomial corresponding
to an automorphism. If that characteristic polynomial is irreducible, we have our
favorable p-group and the necessary automorphism.
Theorem 4.6. A favorable p-group G is a special p-group.
Proof. We already know that G is of class at most 2 and V = G/G is an
elementary-abelian p-group. Let φ be a p -automorphism, such that, if it fixes a
proper subgroup of G, then it is the identity on that subgroup. Since G is charac-
teristic, φ is the identity on G . Consider the module Vφ over Fp [x] corresponding
to φ. Then from the lemma above we know that the characteristic polynomial χφ
is irreducible and Vφ is simple.
In any finite p-group, G ⊆ Φ(G) and from above G ⊆ Z(G). To show G =
Z(G), notice that Vφ is a simple module over Fp [x] and all submodules are φ-
invariant subspaces. So Z(G)/G cannot be a nontrivial submodule. Similar is the
case with Φ(G).
So if we assume that G is not elementary-abelian, then G = Z(G) = Φ(G). •
At this point it is clear, to build a secure and optimal MOR cryptosystem with
non-abelian p-groups one should look at special p-groups and an automorphism φ
such that φ is identity on all subgroup it fixes. In particular φ must centralize Φ(G),
so smaller the Φ(G) the better. So it is clear that we should look for groups with
Φ(G) as small as possible. We conclude that for a non-abelian p-group (p odd) and
p -automorphisms the best group is an extra-special p-group of prime exponent.
For abelian p-groups, we should look only at elementary-abelian p-groups. For p
even, we still have the extra-special groups but we can use any exponent.
5. The MOR cryptosystem and elementary abelian p-group

As is well known, an elementary abelian p-group is a vector space over Fp the
field of p elements. So one way to look at MOR cryptosystems over an elementary
abelian group is MOR cryptosystems over a vector space. If we fix a basis for
the vector space, any linear transformation gives rise to a matrix. So the discrete
logarithm problem in invertible linear transformations turns out to be the discrete
logarithm problem over non-singular matrices. So we need to say a few things about
that. Before we do that, we also need to remind our reader that security and speed
goes hand in hand. One reason, the discrete logarithm problem in matrices was
avoided in cryptography was the belief that matrix exponentiation is much more
expensive. The security advantage we gain from the discrete logarithm problem
in matrices does not outweigh the cost of matrix exponentiation. This view was
put down by Menezes & Wu [13]. However with the recent advances in matrix
exponentiation by Leedham-Green [9], the above argument is no longer valid. We
get into the details of this argument in this section.
5.1. Solving the discrete logarithm problem in non-singular matrices.

Let g and g m belongs to GL(d, q), the discrete logarithm problem is to find m. This
problem can be easy and hard. For uni-triangular matrices, i.e., matrices with one
on the diagonal and arbitrary field element on the upper half and zero on the lower
half, it is very easy. On the other hand, with matrices with irreducible characteristic
polynomial, the discrete logarithm problem is hard.
Following is the work of Menezes & Wu [13], which is the best known algorithm
to solve the discrete logarithm problem in matrices. This algorithm is basically a
reduction of the discrete logarithm problem in GL(d, q) to a finite (possibly trivial)
extension of Fq .
5.2. The Menezes-Wu algorithm.

• Input: g and g m .
• Output: m.
• From g, compute the characteristic polynomial χg of g.
• From g m , compute the characteristic polynomial χgm of g m .
Let {α1 , α2 , . . . , αd } be the characteristic roots of g. This list might contain repeat-
ing entries. The characteristic roots lie in some finite (possibly trivial) extension of
Fq . Let {β1 , β2 , . . . , βd } be the characteristic roots of g m . This list might contain
repeating entries. The roots lie in some finite (possibly trivial) extension of Fq .
Then {β1 , β2 , . . . , βd } is αim1 , αim2 , . . . , αimd , where (i1 , i2 , . . . , id ) is (1, 2, . . . , d)
permuted. Note that there is no obvious way to order characteristic roots, but
following Menezes and Wu, we will assume that this permutation is not going
to offer much resistance in computing m. In other words, we assume that we
can find αi and βj such that αim = βj . Once we have this, one can solve for
m mod o(αi ), where o(αi ) is the multiplicative order of αi . From, solving the
required numbers of discrete logarithm problems in the suitable extensions and
then applying the Chinese remainder theorem, one can solve the discrete logarithm
problem in non-singular matrices. Note that the αi and subsequently the βj will
be in some extension field (possibly trivial) of Fq . The largest extension possible is
Fqd and this happens when the characteristic polynomial is irreducible.
The most serious attack on the discrete logarithm problem in a finite field is
the sub-exponential attack like the index-calculus attack. In this attack, if we are
solving
the discrete logarithm problem
1
in Fqd , the time-complexity of the attack is
2
exp (c + o(1))(log q d ) 3 (log log q d ) 3 , where c is a constant, see [16] and [8, Sec-
tion 4]. It is clear, larger the d more secure is the discrete logarithm problem in
matrices. So we can now safely conclude, to work with the discrete logarithm prob-
lem in matrices one should work with matrices with irreducible characteristic
polynomial.
5.3. Exponentiation in non-singular matrices. This section is a brief in-

troduction to an amazing algorithm by Leedham-Green [9, Section 10] to compute
g m for some g ∈ GL(d, q). We only deal with the case where the characteristic
polynomial χg of g is irreducible.
90 AYAN MAHALANOBIS
Algorithm 5.1 (Leedham-Green).

Input: a matrix g of size d over a finite field Fq and a positive integer m.
Output: g m
• Find a matrix P such that B = P −1 gP is in the Frobenius normal form.
• Determine the minimal polynomial m(x) of B. Since the Smith normal
form is sparse, it is easy to compute the minimal polynomial – it takes
O(d2 ) field multiplications.
• Compute tm mod m(t) in F [t]/m(t) as l(t).
• Compute C = l(B).
• Return P CP −1 .
Notice that the objective of the above algorithm was to compute the power of an
arbitrary matrix. In our case, for a MOR cryptosystem the matrix is not arbitrary,
we can choose our matrix. So one can first choose an irreducible polynomial m of
degree d over Fq . Then choose g to be the companion matrix for that polynomial m.
Since the minimal polynomial divides the characteristic polynomial, the minimal
polynomial is m as well. So the first two steps and the last step in the above
algorithm becomes redundant.
Once m is irreducible in the above algorithm the quotient F[t]/m(t) is a field.
So the third step is essentially an exponentiation in the field Fqd . So apart from
computing the C in the above algorithm, exponentiation of a matrix with irreducible
characteristic polynomial is the same as exponentiation in the finite field Fqd .
The following is now clear: the discrete logarithm problem in GL(d, q) is al-
most the same, both in terms of security and speed, to a discrete logarithm problem
in Fqd . Note that this conclusion is remarkably different than that of Menezes &
Wu [13], where they write-off completely the discrete logarithm problem in matri-
ces.
Next we show that elementary-abelian p-groups are favorable p-groups.
Lemma 5.2. Let V be a vector space over Fp . Let φ be a non-singular linear
transformation on V . If p|o(φ), then V has a proper φ-invariant subspace.
Proof. Let A = φ
. Then the given condition implies that p||A|. Considering
the fact that any finite abelian group is the direct product of its Sylow subgroups,
we see that one can write φ = φp φp . Where φp and φp are p and p non-trivial
automorphism respectively. From the fact that (xq − 1) = (x − 1)q for any p-power
q, we see that all the eigenvalues of φp are 1 ∈ Fp . Let E be the eigenspace of 1
in V . Clearly E is a proper subspace of V . Let v ∈ E. Then φp φp (v) = φp φp (v),
which implies φp φp (v) = φp (v). This proves that φp (v) ∈ E. So E is a φ-invariant
proper subspace of V . •
Theorem 5.3. An elementary-abelian p-group is a favorable p-group.
Proof. An elementary abelian p-group V is a vector space over Fp . Then the
automorphism group of V is GL(V ). Let φ be an automorphism with irreducible
characteristic polynomial. Then φ is a p -automorphism. Then Lemma 4.5 proves
the rest. •
6. The extra-special p-groups and its automorphism group

As we saw before, if we are dealing with p -automorphisms, there are only
two interesting class of finite p-groups. One is the elementary abelian p-group
and the other is extra-special p-groups. The case for extra special p-groups is
interesting, because it provides us with non-abelian p-groups which is presented
in the power-commutator form and provides us with a secure MOR cryptosystem;
thus showing that abstract presentations can be useful. As we will see, the security
with p -automorphisms reduces to the discrete logarithm problem in non-singular
matrices. This enables us to argue that working with p -automorphisms of a p-
group, one has no advantage from working with matrices. However, the case with
p-automorphisms is not quite settled yet. We will see, as an example with the
central automorphisms of the extra-special p-groups that there are some potential
with p-groups. The potential is the impossibility of the reduction to matrices, which
killed the p -automorphisms.
6.1. Extra-special p-groups. It is well known that any special p-group is of
exponent at most p2 . We saw earlier that for odd prime p we can concentrate on
groups of exponent p. So for an odd prime p our principal interest is in the extra-
special p-group of exponent p. Our principal reference is Gorenstein [3, Section
5.5]. We briefly summarize few facts about the extra-special p-group of exponent p
denoted by G.
• The order of G is p2n+1 for some positive integer n. The cardinal-
ity of the minimal set of generators is 2n and let us denote that set
by {x1 , y1 , x2 , y2 , . . . , xn , yn }. There is a relation [xi , yi ] = z, where
Z(G) = z
and z p = 1. Furthermore, [xi , xj ] = 1 and [xi , yj ] = 1
for i = j.
• The group G is the central product of n copies of the group of order p3
given by
x, y, z | xp = y p = z p = 1, [x, z] = 1, [y, z] = 1, [x, y] = z
.
• In the group G, G = G = Φ(G) and is cyclic of order p.
In a p-group, finding all automorphisms is often a very hard job. However, for
an extra-special p-groups it is not that hard. The automorphisms were studied
extensively by Winter [17]. The study of automorphisms of an extra-special p-
group is not that hard because of a bilinear map B : G/G × G/G → Fp . The
map is defined as follows, let x̄, ȳ ∈ G/G , then [x, y] = z a for some integer a.
Then B(x̄, ȳ) = ā, where ā = a mod p. It is known that B is an alternating,
non-degenerate bilinear form on G/G .
We will not do a detailed presentation of the automorphisms of the extra-special
p-group of prime exponent. An interested reader can find that in Winter [17].
However, to facilitate further discussion we have to describe them briefly.
Since an extra-special p group is of class 2, we have that [xn , y] = [x, y]n . Recall
that the center Z(G) is of prime order and any automorphism of Z(G) can be lifted
to an automorphism of G. So we have a complete description of the automorphisms
of G, that are not identity on Z(G).
So now we have to concentrate on the automorphisms that fix Z(G). It was
shown by Winter that an automorphism φ of G is an automorphism of G/Z(G) if
and only if it is the identity on Z(G).
It was further shown that for prime exponent, the automorphisms that fix Z(G)
is the symplectic group Sp(2n, p). Winter denotes this subgroup of the automor-
phism group by H and has shown that it is a normal subgroup of the automorphism
group.
92 AYAN MAHALANOBIS
To summarize, there are two kinds of automorphisms:

a: Automorphisms that are not the identity on the center Z(G) of G. Since,
any automorphism of the center can be extended to an automorphism of
the whole group, and the center is cyclic. We have a complete under-
standing of these automorphisms. They are uninteresting to our cause.
b: One that are identity on the center. These automorphisms form a normal
subgroup of the automorphism group of G. We will call them H.
For obvious reasons we are interested in b above. Let φ be an automorphism
that centralizes the center. Winter has shown that, φ̄ : G/Z(G) → G/Z(G) is
an automorphism of G/Z(G) preserving the bilinear form B. We will abuse the
notation a little bit and call the automorphism on the central quotient φ as well.
An interesting normal subgroup of H is the group of inner automorphisms I.
Using the fact that the commutator G ⊆ Z(G) and the identity ab = ba[a, b] for
any a, b ∈ G, it is clear that an inner automorphism is of the form
xi →
xi z di

yi → yi z di where 0 ≤ di , di < p.
From the fact, the group of the inner automorphisms I is isomorphic to G/Z(G),
it follows that there are p2n inner automorphisms. It also follow from a simple
counting argument on all possible choices of di and di . From our understanding of
the inner automorphisms, the following proposition is clear:
Proposition 6.1. An automorphism φ of G is an inner automorphism if and
only if it is the identity on Z(G) and G/Z(G). The inner automorphisms commute
and constitutes the group of central automorphisms.
It is known [17, 3E], H/I is isomorphic to Sp(2n, p). Recall that G/Z(G) is
a symplectic vector space over Fp . We next show that the extra-special p-group of
prime exponent is a favorable p-group.
Theorem 6.2. For an odd prime p, the extra-special p group of exponent p is
a favorable p-group.
Proof. Let φ ∈ Sp(2n, p), such that χφ is irreducible. From the above dis-
cussion, we can consider φ to be an automorphism of G that is the identity on
G . According to Lemma 4.5, there are no proper φ-invariant subspaces of G/G ,
and from Lemma 5.2 φ is a p -automorphism. Now assume that H is a proper
φ-invariant subspace of G. Then consider HG . Notice that G = Φ(G) and fur-
thermore Φ(G) is the set of non-generators of G. Then it follows that HG is a
proper subgroup and so HG /G is a proper φ-invariant subspace of G/G . Which
implies that HG ⊆ G and furthermore H ⊆ G . •
Corollary 6.3. For an odd prime p, the extra-special p-group of exponent p
is self-critical.
Proof. Let G denote the extra-special p-group of exponent p and C be a
critical subgroup of G. Then the condition CG (C) = Z(G) implies that C is not
contained in Z(G). From the above theorem G is a favorable p-group. Then there
is a corresponding automorphism φ. Let V = G/G and construct Vφ and it is
known to be simple. Consider the subgroup CG . Then CG is either the whole
group or the center Z(G). Since it can’t be Z(G), it is the whole group. Now
notice that G = Φ(G) and Φ(G) is the set of non-generators of G. It follows that
if CG = G, C = G. So G is self-critical. •
6.2. The case when p = 2. In this case a theorem of Winter [17, Theorem
1(c)] comes in handy.
Theorem 6.4. Let P be an extra-special group of order 22n+1 . Subgroups H
and I are as defined earlier. Then H/I is isomorphic to the orthogonal group

2i
n−1
Oε (2n, 2) of order 2n(n−1)+1 (2n − ε) 2 − 1 . Here, ε = 1 if P is isomorphic
i=1
to the central product of n dihedral groups of order 8 and ε = −1 if P is isomorphic
to the central product of n − 1 dihedral group of order 8 and a quaternion group.
From the above theorem, by selecting appropriate matrix with irreducible char-
acteristic polynomial, it is easy to see that the case p = 2 follows the exact same
pattern as that of p = 2. So we won’t dwell with p = 2 any further.
7. MOR cryptosystems on finite p-groups using p-automorphisms

In the last section we looked at p -automorphisms. In this section, we look at
p-automorphisms. Our standard reference for p-automorphisms is Khukhro [7].
To recall, we looked at the exponent-p central series of a p-group. It is known
that this series has elementary abelian sections. There are two cases with p-
automorphisms.
a: The automorphism φ is not identity on at least one section of the series.
b: The automorphism φ is identity on all the sections.
In the case a above, one can not build a secure MOR cryptosystem. The reason
is as follows:
Theorem 7.1. Let V be a vector space over Fq , a field of characteristic p > 0.
Let φ be a p-automorphism. Then φ can be written as a block-diagonal matrix with
1 in the diagonal. Phrased differently, all the eigenvalues of φ are 1.
Proof. The theorem is well-known, see [7, Theorem 2.5]. •
Once we have this theorem, the fact that the discrete logarithm problem in
that matrix is easy follows from the following observation and the fact that the
power of a block diagonal is the power of the respective blocks written as a block
diagonal matrix maintaining the order of the block:
⎛ ⎞m ⎛ ⎞
1 1 ∗ ... ∗ 1 m ∗ ... ∗
⎜0 1 1 . . . 0⎟ ⎜0 1 m . . . 0⎟
⎜ ⎟ ⎜ ⎟
⎜ .. .. .. . ⎟ = ⎜ . .. .. .⎟ .
⎝. 0 . . .. ⎠ ⎝ .. 0 . . .. ⎠
0 ... ... 0 1 0 ... ... 0 1
This proves that the case a above is useless.
However, the case b above is of immense interest to us. We will give an example
of this kind of automorphism. The reason for immense interest is as follows: anyone
who is trying to build a new cryptosystem, will want to build a new cryptosystem.
In the case of p -automorphisms, in the MOR cryptosystem we saw, the security
can be reduced to that of the discrete logarithm problem in matrices. The discrete
logarithm problem in matrices is not a new cryptographic primitive. In this case
(b above) we have a real good possibility of a new cryptographic primitive.
94 AYAN MAHALANOBIS
Let us look at the situation in some details. There are two subgroups of the
automorphism group that we are interested in. One is the group of central auto-
morphisms and the other is the group of inner automorphisms.
7.1. Central automorphisms. Most central automorphisms are p-automor-
phisms. To quote Curran and McCaughan [2], “So, roughly speaking, most of the
central automorphisms are of p-power order”.
Central automorphisms are the centralizer of the group of inner automorphisms
in the automorphism group, they form a normal subgroup in the automorphism
group. Let φ be a central automorphism, then φ(g) = gzg , zg ∈ Z(G). It is
clear from the definition that central automorphisms centralize the commutator
subgroup. Now take an example of a finite p-group G, such that Z(G) ⊆ G . In
this group, for a g ∈ G, we have φ(g) = gzg and φm (g) = gzgm . So from g −1 φ(g)
and g −1 φm (g), the discrete logarithm problem in the automorphism φ reduces to
the discrete logarithm problem in zg ∈ Z(G). This is exactly the case with the
extra-special p-group (see Proposition 6.1). In the case of the extra-special p-group
of prime exponent, a central automorphisms acts as the identity in both Z(G) and
G/Z(G). So the obvious way to reduce an automorphism to matrices over Fp do
not work. However in this case, as demonstrated earlier, it reduces to the discrete
logarithm problem in the center. The open question is, can there be other (secure)
situations in which the discrete logarithm problem in the automorphism is not the
discrete logarithm problem in the usual sense?
7.2. Inner automorphisms. The group of inner automorphisms of a p-group
G is a p-group. Let G = G1 G2 . . . Gk = 1 be a sequence of subgroups in a
p-group G. Let g ∈ CG (G2 ) be an element. Then consider the inner automorphism
φ such that φ(x) = g −1 xg. Then clearly, φ acts as the identity on Gi for i ≥ 2
and Gi /Gi+1 for i ≥ 1. However, this is not enough. Recall that our target is,
φ should act like the identity on all possible sections H/K where φ fixes K and
H/K is elementary-abelian. The question is, are there p-groups, on which, using
the inner automorphisms, one can build a secure MOR cryptosystem?
8. Conclusion
This paper is a study of finite p-groups for the MOR cryptosystem. The aim of
this paper was not to provide with a secure MOR cryptosystem. For that, one can
look into the arXiv preprint [10]. The purpose of this paper is to theoretically justify
what can one expect out of finite p-groups. There are two classes of automorphisms
one should look at. One is p-automorphisms and the other is p -automorphisms.
The case of p -automorphism has been resolved in this paper as follows: for abelian
groups, it is the elementary-abelian p-groups. For non-abelian groups, one should
use the extra-special p-groups of exponent p. However there are very interesting
questions that are open for p-automorphisms. We point those out in this paper.
References
[1] Yakov Berkovich, Groups of prime power order. Vol. 1, de Gruyter Expositions in Mathemat-
ics, vol. 46, Walter de Gruyter GmbH & Co. KG, Berlin, 2008. With a foreword by Zvonimir
Janko. MR2464640 (2009m:20026a)
[2] M. J. Curran and D. J. McCaughan, Central automorphisms of finite groups, Bull. Aus-
tral. Math. Soc. 34 (1986), no. 2, 191–198, DOI 10.1017/S0004972700010054. MR854565
(87k:20042)
[3] Daniel Gorenstein, Finite groups, 2nd ed. Chelsea Publishing Co., New York, 1980.
MR569209 (81b:20002)
[4] P. Hall and Graham Higman, On the p-length of p-soluble groups and reduction theorems for
Burnside’s problem, Proc. London Math. Soc. (3) 6 (1956), 1–42. MR0072872 (17,344b)
[5] Jeffrey Hoffstein, Jill Pipher, and Joseph H. Silverman, An introduction to mathematical
cryptography, Undergraduate Texts in Mathematics, Springer, New York, 2008. MR2433856
(2009m:94051)
[6] B. Huppert and N. Blackburn, Finite Groups II, Springer-Verlag, 1982.
[7] E. I. Khukhro, p-automorphisms of finite p-groups, London Mathematical Society Lecture
Note Series, vol. 246, Cambridge University Press, Cambridge, 1998. MR1615819 (99d:20029)
[8] Neal Koblitz, Alfred Menezes, and Scott Vanstone, The state of elliptic curve cryptogra-
phy, Des. Codes Cryptogr. 19 (2000), no. 2-3, 173–193, DOI 10.1023/A:1008354106356.
MR1759616 (2001i:94065)
[9] C. R. Leedham-Green and E. A. O’Brien, Constructive recognition of classical groups in odd
characteristic, J. Algebra 322 (2009), no. 3, 833–881, DOI 10.1016/j.jalgebra.2009.04.028.
MR2531225 (2010e:20075)
[10] Ayan Mahalanobis, The MOR cryptosystem and extra-special p-groups, http://arxiv.org/
abs/1111.1043.
[11] , A simple generalization of the ElGamal cryptosystem to non-abelian groups, Com-
munications in Algebra 36 (2008), no. 10, 3880–3891.
[12] Ayan Mahalanobis, A simple generalization of the ElGamal cryptosystem to non-abelian
groups II, Comm. Algebra 40 (2012), no. 9, 3583–3596, DOI 10.1080/00927872.2011.602998.
MR2981154
[13] Alfred J. Menezes and Yi-Hong Wu, The discrete logarithm problem in GL(n, q), Ars Combin.
47 (1997), 23–32. MR1487162 (98j:11122)
[14] M. F. Newman, Werner Nickel, and Alice C. Niemeyer, Descriptions of groups of prime-
power order, J. Symbolic Comput. 25 (1998), no. 5, 665–682, DOI 10.1006/jsco.1997.0193.
MR1617995 (99f:20054)
[15] Steven Roman, Advanced linear algebra, 3rd ed. Graduate Texts in Mathematics, vol. 135,
Springer, New York, 2008. MR2344656 (2008f:15002)
[16] Oliver Schirokauer, Damian Weber, and Thomas Denny, Discrete logarithms: the effectiveness
of the index calculus method, Algorithmic number theory (Talence, 1996), Lecture Notes in
Comput. Sci. vol. 1122, Springer, Berlin, 1996, pp. 337–361, DOI 10.1007/3-540-61581-4 66.
MR1446523 (98i:11109)
[17] David L. Winter, The automorphism group of an extraspecial p-group, Rocky Mountain J.
Math. 2 (1972), no. 2, 159–168. MR0297859 (45 #6911)
IISER Pune, Dr. Homi Bhabha Road, Pashan Pune-411008, India

E-mail address: ayan.mahalanobis@gmail.com
Volume 633, 2015
A group theoretical ElGamal cryptosystem

based on a semidirect product of groups
and a proposal for a signature protocol
Anja I. S. Moldenhauer
Abstract. In this paper a group theoretical ElGamal cryptosystem is intro-
duced, which is based on a semidirect product of groups. It is developed from
the key exchange protocol based on a semidirect product of (semi)groups intro-
duced by M. Habeeb, D. Kahrobaei, C. Koupparis and V. Shpilrain. Finally,
a proposal for a signature protocol is described.
1. Introduction
In this Section we state definitions and introduce notation that will later on
be used. First the definition of a semidirect product and the extension by au-
tomorphisms is recalled (see [HKKS13]). After that the classical Diffie-Hellman
key exchange protocol and the consequential ElGamal public key cryptosystem are
introduced ([MSU08, Section 1.2 and 1.3] are used as an orientation).
Definition 1.1. Let G, H be two groups, let Aut(G) be the group of automor-
phisms of G and let ρ : H → Aut(G) be a homomorphism. Then the semidirect
product of G and H is the set
Γ = G ρ H = {(g, h) | g ∈ G, h ∈ H}
with the group operation given by

(g, h) · (g , h ) = (g ρ(h ) · g , h · h ).

Here g ρ(h ) denotes the image of g under the automorphism ρ(h ), and a product
h · h of two morphisms means that h is applied first.
1.1. Extension by automorphisms. One special case of the semidirect
product construction is where the group H is a subgroup of the group Aut(G). If
H = Aut(G), then the corresponding semidirect product is called the holomorph
of the group G. Thus, the holomorph of G, usually denoted by Hol(G), is the set
Hol(G) = {(g, φ) | g ∈ G, φ ∈ Aut(G)}
2010 Mathematics Subject Classification. Primary 94A60; Secondary 11T71, 20F05, 20K25.
Key words and phrases. Semidirect products of groups, algebraic cryptography, signature
protocol.
2015
97
98 ANJA I. S. MOLDENHAUER
with the group operation given by

(g, φ) · (g , φ ) = (φ (g) · g , φ · φ ).
It is often more practical to use a subgroup of Aut(G) in this construction, as it is

done in [HKKS13, Section 3], where a key exchange protocol is described, that uses
(as the platform) an extension of a group G by a cyclic group of automorphisms.
This key exchange is described in more details in Section 2.
Remark 1.2. This construction is also used if G is not necessarily a group,

but just a semigroup, and/or if endomorphisms of G, that are not necessarily au-
tomorphisms of G, are considered. Then the result will be a semigroup.
1.2. The Diffie-Hellman key establishment. The simplest, and original,

implementation of the protocol of W. Diffie and M. E. Hellman (see [DH79] or
[MSU08, Section 1.2]) uses the multiplicative group of integers modulo p, where p
is prime and g is primitive mod p. A more general description of the protocol uses
an arbitrary finite cyclic group.
(1) Alice and Bob agree on a finite cyclic group G and a generating element
g in G. The group G is written multiplicatively.
(2) Alice picks a random natural number a and sends g a to Bob.
(3) Bob picks a random natural
b a number b and sends g b to Alice.
ba
(4) Alice computes KA = g =g .
a b
(5) Bob computes KB = (g ) = g ab .
Since ab = ba (because N is commutative), both, Alice and Bob, are now in pos-
session of the same group element K = KA = KB which can serve as the shared
secret key.
The Diffie-Hellman key exchange is summarized in Table 1.
Table 1. Diffie-Hellman key exchange.
Public Parameters
Finite cyclic group G and
a generating element g ∈ G. Write G multiplicatively.
Alice Bob
Pick a ∈ N randomly. Pick b ∈ N randomly.
Compute A := g a . Compute B := g b .
−−−−−−−−−−−−−−→
A
←−−−−−−−−−−−−−−
−
B
a
Compute KA := B a = g b = g ba . Compute KB := Ab = (g a )b = g ab .
K = KB = KA
The protocol is considered secure against eavesdroppers if G and g are cho-

sen properly. The eavesdropper, Eve, must solve the Diffie-Hellman problem
(recover g ab from g a and g b ) to obtain the shared secret key. This is currently
considered difficult for a “good” choice of parameters (see e.g. [MvOV97] for
details.)
GROUP THEORETICAL ELGAMAL CRYPTOSYSTEM 99
1.3. ElGamal cryptosystem. The ElGamal cryptosystem (see [ElG85] or

[MSU08, Section 1.3]) is a public key cryptosystem which is based on the Diffie-
Hellman key establishment (see Section 1.2).
(1) Alice and Bob agree on a finite cyclic group G and a generating element
g ∈ G.
(2) Alice (the receiver) picks a random natural number a and publishes the
element c := g a .
(3) Bob (the sender), who wants to send a message m ∈ G (also called “plain-
text”) to Alice, picks a random natural number b and sends two elements,
m · cb and g b , to Alice. Note that cb = g ab .
a −1
(4) Alice recovers m = m · cb · g b .
A notable feature of the ElGamal encryption is that it is probabilistic, mean-
ing that a single plaintext can be encrypted to many possible ciphertexts.
The ElGamal cryptosystem is summarized in Table 2.
Table 2. ElGamal cryptosystem.
Public Parameters
Finite cyclic group G and
a generating element g ∈ G. Write G multiplicatively.
Alice Bob
Key Creation
Pick private a ∈ N randomly.
Publish c := g a .
Encryption
Choose plaintext m ∈ G.
Pick b ∈ N randomly.
Compute c1 := m · cb and c2 := g b .
Send (c1 , c2 ) to Alice.
Decryption
Recover
a −1
m = c1 · ((c2 )a )−1 = m · cb · g b
= m · g ab · g −ba .
Remark 1.3. The ElGamal encryption has an average expansion factor of 2,

i.e., the encryption length of a message is the double of the message itself.
2. Key exchange protocol based on a semidirect product of

(semi)groups
In this Section, the new key exchange protocol based on a semidirect product
of (semi)groups by automorphisms from M. Habeeb, D. Kahrobaei, C. Koupparis
and V. Shpilrain (see [HKKS13]) is described.
Alice and Bob use a group (or semigroup) G and they can use just a cyclic
subgroup H (or a cyclic subsemigroup) of the group Aut(G) (respectively, of the
semigroup End(G) of endomorphisms) instead of the whole group of automorphisms
of G.
Let G be a (semi)group. An element g ∈ G as well as an arbitrary automor-
phism φ ∈ Aut(G) (or an arbitrary endomorphism φ ∈ End(G)) are chosen and
published.
Both, Alice and Bob, are going to work with elements of the form (g, φr ),
where g ∈ G and r ∈ N. Note that two elements of this form are multiplied as
follows:

(g, φr ) · (h, φs ) = φs (g) · h, φr+s .
(1) Alice chooses a private m ∈ N.

She computes (g, φ)m = (φm−1 (g)·φm−2 (g)·. . .·φ(g)·g, φm ) and sends
only the first component, namely a := φm−1 (g) · φm−2 (g) · . . . · φ(g) · g, to
Bob.
(2) Bob chooses a private n ∈ N.
He computes (g, φ)n = (φn−1 (g) · φn−2 (g) · . . . · φ(g) · g, φn ) and sends
only the first component, namely b := φn−1 (g) · φn−2 (g) · . . . · φ(g) · g, to
Alice.
(3) Alice computes (b, x) · (a, φm ) = (φm (b) · a, x · φm ).
Her key is now KA := φm (b) · a. Note that she does not actually
“compute” x · φm because she does not know the automorphism x = φn ;
recall that it was not transmitted to her. But she does not need it to
compute KA .
(4) Bob computes (a, y) · (b, φn ) = (φn (a) · b, y · φn ).
His key is now KB := φn (a) · b. Again, Bob does not actually “com-
pute” y · φn because he does not know the automorphism y = φm .
(5) Since (b, x) · (a, φm ) = (a, y) · (b, φn ) = (g, φ)m+n ,
it should be KA = KB = K, the shared secret key.
Remark 2.1. The shared secret key is K = KB = KA , because
KB = φn (a) · b
= φn (φm−1 (g) · φm−2 (g) · . . . · φ(g) · g) · φn−1 (g) · φn−2 (g) · . . . · φ(g) · g
= φn+m−1 (g) · φn+m−2 (g) · . . . · φn+1 (g) · φn (g) · φn−1 (g) · φn−2 (g) · . . . · φ(g) · g
= φm (φn−1 (g) · φn−2 (g) · . . . · φ(g) · g) · φm−1 (g) · φm−2 (g) · . . . · φ(g) · g
= φm (b) · a
= KA .
The cost of computing (g, φ)n is O(log n) (see [HKKS13]) just as in the standard
Diffie-Hellman protocol.
Remark 2.2. In contrast to the standard Diffie-Hellman key exchange, the

correctness here is based on the equality hm · hn = hn · hm = hm+n rather than
n m
on the equality (hm ) = (hn ) = hmn . In the standard Diffie-Hellman set up,
the trick would not work, because, if the shared key K was just the product of
two openly transmitted elements, then anybody, including the eavesdropper, could
compute K.
The key exchange protocol using semidirect product of (semi)groups is sum-

marized in Table 3.
Table 3. Key exchange protocol using semidirect product of (semi)groups.
Public Parameters
G (semi)group, H cyclic sub(semi)group of the group Aut(G) (or End(G)),
φ ∈ H ⊆ Aut(G) (respectively φ ∈ H ⊆ End(G)) and an element g ∈ G.
Alice Bob
Choose private m ∈ N. Choose private n ∈ N.
Compute Compute
(a, φm ) := (g, φ)m (b, φn ) := (g, φ)n
with a := φm−1 (g) · φm−2 (g) · . . . · φ(g) · g. with b := φn−1 (g) · φn−2 (g) · . . . · φ(g) · g.
−−−−a−−−−−−−−−−→
←−−−−−−−−−−−−−−
b
Compute Compute
(b, x) · (a, φm ) = (φm (b) · a, x · φm ). (a, y) · (b, φn ) = (φn (a) · b, y · φn ).

=:KA =:KB
K = KB = KA
3. The MR public key cryptosystem

The public key cryptosystem, which is presented in this Section, develops from
an idea from Gerhard Rosenberger, therefore it is called the MR public key cryp-
tosystem. It is an ElGamal-like cryptosystem and it is based on the semidirect
product of groups. After the general description of the MR public key cryptosys-
tem we give two examples for possible platform groups and discuss their security.
Alice and Bob can use a group G and a cyclic subgroup H of the group Aut(G)
instead of the whole group of automorphisms of G.
(1) Alice and Bob agree on an element g ∈ G and an automorphism
φ ∈ H ⊆ Aut(G). Whereby, they have to take care, that the base element
(g, φ) has a large order, otherwise the system is susceptible to brute force
attacks.
(2) Alice chooses a random natural number n as her secret key.
She computes (g, φ)n = (φn−1 (g) · φn−2 (g) · . . . · φ(g) · g, φn ) and publishes
the first component a := φn−1 (g) · φn−2 (g) · . . . · φ(g) · g only.
(3) Bob wants to send the message m ∈ G to Alice. He picks a random
ephemeral key r ∈ N. Therefore he has to calculate two elements.
He computes (g, φ)r = (φr−1 (g) · φr−2 (g) · . . . · φ(g) · g, φr ) whichs first
component is named c1 := φr−1 (g) · φr−2 (g) · . . . · φ(g) · g. Then he
computes (a, y) · (c1 , φr ) = (φr (a) · c1 , y · φr ). He sets the first component
b := φr (a) · c1 . Note that he does not actually “compute” y · φr , because
he does not know the automorphism y = φn , but he does not need it
to compute b. He computes c2 := b · m = φr (a) · c1 · m and sends the
ciphertext (c1 , c2 ) to Alice.
(4) Alice computes (c1 , x) · (a, φn ) = (φn (c1 ) · a, x · φn ), named the first com-
ponent K := φn (c1 ) · a and recovers m = K −1 · c2 = (φn (c1 ) · a)−1 · c2 .
Note that she does not “compute” x·φn because she does not know x = φr
and does not need it to compute K.
Alice gets the message m, because from

K −1 · c2 = (φn (c1 ) · a)−1 · c2 = (φn (c1 ) · a)−1 · φr (a) · c1 · m
with
φn (c1 ) · a = φr (a) · c1 ,
which follows from the same calculations as in Remark 2.1, it is
K −1 · c2 = (φn (c1 ) · a)−1 · c2
= (φn (c1 ) · a)−1 · φr (a) · c1 · m
= (φn (c1 ) · a)−1 · φn (c1 ) · a · m
= m.
The MR public key cryptosystem is summarized in Table 4.
Table 4. The MR public key cryptosystem.
Public Parameters
Group G and cyclic subgroup H of the group Aut(G),
g ∈ G and φ ∈ H ⊆ Aut(G).
Alice Bob
Key Creation
Choose private key n ∈ N.
Compute
(a, φn ) := (g, φ)n
with a := φn−1 (g) · φn−2 (g) · . . . · φ(g) · g.
Publish a.
Encryption
Choose random ephemeral key r ∈ N.
Compute
(c1 , φr ) := (g, φ)r
with c1 := φr−1 (g) · φr−2 (g) · . . . · φ(g) · g,
(a, y) · (c1 , φr ) = (φr (a) · c1 , y · φr )

=:b
and
c2 := b · m = φr (a) · c1 · m.
Send ciphertext (c1 , c2 ) to Alice.
Decryption
Compute
(c1 , x) · (a, φn ) = (φn (c1 ) · a, x · φn )

=:K
and recover
m = K −1 · c2 .
Remark 3.1. Alice computes a large power of the element (g, φ), but she
does not transmit the whole result, she only publishes the part a of it. Bob also
computes a large power of the element (g, φ) and only the first part c1 is a part
of his ciphertext. In addition, he computes a product of two elements from G and
only the first part multiplied by the message is the second part of his ciphertext.
It is important that random ephemeral keys r are used to encrypt different
messages. As it is for the standard ElGamal cryptosystem (see [MvOV97]). Sup-
pose that Bob uses the same ephemeral key r to encrypt two messages m1 and m2
and assume that m1 is known. The ciphertext pairs are (c1 , c2 ) and (c1 , c2 ), with
c1 = c1 , c2 = φr (a) · c1 · m1 and c2 = φr (a) · c1 · m2 . Eve only has to calculate
m1 · (c2 )−1 · c2 to get the message m2 .
Another non-commutative generalization of the ElGamal key exchange which
is based on the complexity differences between various group-theoretic decision
problems and uses polycyclic groups can be found in [KK06].
3.1. Example for the MR public key cryptosystem with G = Z∗p . Fol-
lowing the example that has been shown in [HKKS13, Section 5] for the key
exchange presented there, we now use the multiplicative group Z∗p as the platform
group G for illustration purposes.
Let G be the multiplicative group Z∗p with p prime.

For the endomorphisms φ of the group Z∗p a number k ∈ N, k > 1, is selected,
such that
φ(h) = hk for every h ∈ Z∗p .
If k is relatively prime to p − 1, then φ is actually an automorphism.
For an element g ∈ Z∗p and m ∈ N it is
(g, φ)m = (φm−1 (g) · φm−2 (g) · . . . · φ(g) · g, φm )
with
m−1 m−2
φm−1 (g) · φm−2 (g) · . . . · φ(g) · g = g k · gk · . . . · gk · g
m−1
+km−2 +...+k+1
= gk
km −1
=g k−1 ,
because the finite geometric sum is used and
r
φr (g) = g k for all r ∈ N.
An example is performed in Table 5.
3.2. Security of the MR public key cryptosystem with the platform
group G = Z∗p . If the eavesdropper Eve wants to get the message m by calculation
kr+n −1
b−1 · c2 = b−1 · g k−1 ·m = m

=b
she has to know the “key” b.

On the one hand she can compute b in two ways by solving the discrete loga-
rithm problem. First she can compute b = φn (c1 ) · a. For this she needs the private
key n from Alice. As an alternative she computes b = φr (a) · c1 . For this she has
to get the ephemeral key r from Bob. In both ways she has to solve the discrete
logarithm problem twice. For example, if she wants to get the private ephemeral
kr −1
r
−1
key r from Bob she first has to recover kk−1 from c1 := g k−1 , and then she has to
recover r from kr , because k is known since φ is published.
On the other hand she can recover b by the analog of what is called the Diffie-
kr+n −1
Hellman problem, so she should recover b := g k−1 from the triple
kr −1 kn −1

g, c1 := g k−1 , a := g k−1 .
Table 5. Example with G = Z∗p .
Public Parameters
G = Z∗p with p prime, φ(h) = hk ∀h ∈ Z∗p
with qualified k ∈ N, k > 1, and g ∈ Z∗p .
Alice Bob
Key Creation
Compute
(a, φn ) := (g, φ)n
with a := φn−1 (g) · φn−2 (g) · . . . · φ(g) · g
kn −1
=g k−1 .
Publish a.
Encryption
Choose plaintext m ∈ Z∗p .
Choose random ephemeral key r ∈ N.
Compute
(c1 , φr ) := (g, φ)r
with c1 := φr−1 (g) · φr−2 (g) · . . . · φ(g) · g,
(a, y) · (c1 , φr ) = (φr (a) · c1 , y · φr )

=:b
and
c2 := b · m = φr (a) · c1 · m.
kr −1
Send c1 = g k−1 and
· c1 · m
c2 = φr (a)
kn −1 kr −1
= φr g k−1 · g k−1 · m
n k r
k −1 kr −1
= g k−1 · g k−1 · m
kr+n −1
= g k−1 · m
as ciphertext (c1 , c2 ) to Alice .
Decryption
Compute
(c1 , x) · (a, φn ) = (φn (c1 ) · a, x · φn ),

=:K
it is
kr −1 kn −1
K = φn g k−1 · g k−1

k n
kr −1 kn −1
= g k−1 · g k−1
kr+n −1
= g k−1 .
Recover
m = K −1 · c2
−kr+n +1 kr+n −1
=g k−1 ·g k−1 · m.
This is exactly the Diffie-Hellman problem, because Eve knows the elements g and
kn+r
k, which
kn
are
kr
public parameters, and it is equivalent to recover g from the triple
g, g , g .
If the group G is the multiplicative group Z∗p , with p prime, then our protocol
is not really different from the standard ElGamal cryptosystem.
Therefore, the standard ElGamal cryptosystem is a special case of the MR cryp-

tosystem, hence, breaking the MR cryptosystem would imply breaking the ElGamal
cryptosystem.
3.3. Example for the MR public key cryptosystem with a non-com-
mutative group. Choose a non-commutative group G, not a semigroup, because
the inverse of an element g −k−n hk+n with g, h ∈ G is needed. For example G =
GL(r, K) with r ∈ N, r > 1, and a field K, the general linear group of r × r matrices
with entries from a field.
Use an extension of the group G by an inner automorphism ρH which is con-
jugation by a matrix H ∈ GL(r, K). Alice and Bob can use any non-commutative
group G if ρH is selected to be a non-trivial inner automorphism, i.e., a conjugation
by an element which is not in the center of G, where the center of GL(r, K) is the
set defined as
C(GL(r, K)) = {α · I|α ∈ K \ {0} and I the identity matrix in GL(r, K)}.
For any Matrix M ∈ G and for any k ∈ N, k > 0, it is
ρH (M ) = H −1 M H and ρkH (M ) = H −k M H k .
For s ∈ N, s > 0, it is
(M, ρH )s = (H −(s−1) M H s−1 · H −(s−2) M H s−2 · . . . · H −1 M H · M, ρsH )
= (H −s (HM )s , ρsH ).
An example is performed in Table 6.
Remark 3.2. If the matrices H and HM commute, Eve can use c1 and c2 to
get the element
V := c−1
1 · c2 = (HM )
−k k
H · H −k−n (HM )n+k · m = H −n (HM )n · m.
The public key is the element a = H −n (HM )n and hence everyone could compute
m in the following way:
a−1 · V = a−1 c−1
1 · c2
= (H −n (HM )n )−1 H −n (HM )n · m = m.

The inverse of a and c1 exist because G is a group. To prevent this Alice has to
take care that H and HM do not commute.
3.4. Security of the MR public key cryptosystem with the platform
group G = GL(r, K). As in Section 3.2 with the platform group G = Z∗p the
eavesdropper Eve can get the message m if she is aware of the “key” b, it is
b = H −(n+k) (HM )n+k . She then calculates
b−1 · c2 = b−1 · H −(n+k) (HM )n+k ·m = m.

=b
For example she can get b by computing b = ρkH (a) · c1 . Therefore she has to try to
recover the ephemeral key k from Bob, i.e., she has to recover k from the element
c1 := H −k (HM )k = g −k hk (with g := H and h := HM ). In the special case with
g = I it is the discrete logarithm problem for matrices in GL(r, K), recover k
from hk . It is known (see [MW97]) that a probabilistic polynomial-time reduction
of the discrete logarithm problem exists in the general linear group GL(r, q) (r × r
matrices with entries of a finite field with q elements) to the discrete logarithm
Table 6. Example with G = GL(r, K).
Public Parameters
Group G = GL(r, K), r ∈ N and r > 1, a matrix H ∈ G, therefore the automorphism is ρH ,
and a matrix M ∈ G. Take care that H and HM do not commute.
Alice Bob
Key Creation
Compute
(a, ρn
H ) := (M, ρH )
n
with a := ρn−1
H (M ) · ρn−2
H (M ) · . . . · ρH (M ) · M
−n
= H (HM )n .
Publish a.
Encryption
Choose random ephemeral key k ∈ N.
Compute
(c1 , ρkH ) := (M, ρH )k with
k−1 k−2
c1 := ρH (M ) · ρH (M ) · . . . · ρH (M ) · M ,
(a, y) · (c1 , ρkH ) = (ρkH (a) · c1 , y · ρkH )

=:b
and
c2 := b · m = ρkH (a) · c1 · m.
Send c1 = H −k (HM )k and
c2 = ρkH (a) · c1 · m
= H −k H −n (HM )n H k · H −k (HM )k · m
= H −k−n (HM )n+k · m
as ciphertext (c1 , c2 ) to Alice.
Decryption
Compute
(c1 , x) · (a, φn ) = (φn (c1 ) · a, x · φn ),

=:K
it is
K = φn (c1 ) · a
= H −n H −k (HM )k H n · H −n (HM )n
= H −n−k (HM )k+n .
Recover
m = K −1 · c2
= (H −n−k (HM )k+n )−1 · H −k−n (HM )n+k · m.
problem in some small extension fields of Fq (a finite field of order q, with q = ps

where p is the characteristic of Fq ). Statistical experiments show that for a random
matrix M , matrices M n are indistinguishable from random (see [HKKS13]).
Furthermore, the security assumption is that it is computationally hard to reclaim
the “key” b = H −(n+k) (HM )n+k from the quadruple

H, M, a := H −n (HM )n , c1 := H −k (HM )k .
Therefore Alice has to take care that the matrices H and HM do not commute
(see Remark 3.2).
4. Signature with a semigroup of 3 × 3 matrices over Z7 [A5 ]

In this Section an idea for a signature scheme inspired by the example of the
key exchange protocol with a semigroup as platform group (see [HKKS13, Section
6]) is described and a security analysis is given.
In [KK12] there is a survey about several digital signature proposals using
non-commutative groups and rings.
Let G be a non-commutative semigroup which has non-central invertible el-
ements, the ρH is a non-identical inner automorphism, i.e., a conjugation by an
element H ∈ G such that H −1 gH = g for at least some g ∈ G.
(1) Alice chooses an invertible H ∈ G for the automorphism ρH and a qualified
hash function h, with
h : {possible messages} −→ {non-invertible matrices from G}
(see Section 4.1 (II) and (III)). This is published.
(2) Alice picks n ∈ N and an element M ∈ G private.
She computes (M, ρH )n = (ρn−1 H (M ) · ρH (M ) · . . . · ρH (M ) · M, ρH ) and
n−2 n
n−1 n−2
publishes only the first component a := ρH (M )·ρH (M )·. . .·ρH (M )·M .
Alice has to take care that H and HM do not commute (see Remark 4.2)
and that her element a has no inverse in G (see Section 4.1 (I)).
(3) To sign the message m she picks an ephemeral key k ∈ N, and computes
(M, ρH )k = (ρk−1 H (M ) · ρH (M ) · . . . · ρH (M ) · M, ρH ) with the first com-
k−2 k
ponent named b := ρH (M ) · ρH (M ) · . . . · ρH (M ) · M . With the help

k−1 k−2
of the hash function h she computes the element Z := h(m) · ρnH (b). Her
signature is the quadruple (k, b, Z, m).
(4) Before Bob can prove the signature he has to calculate the element
(a, x) · (b, ρkH ) = (ρkH (a) · b, x · ρkH ). The first component is named
E := ρkH (a) · b. Note that he does not actually “compute” x · ρkH because
he does not know the automorphism x = ρnH , but he does not need it
to compute E. Bob is aware of the hash function h and he proves the
signature with the calculation Z · a = h(m) · E.
It is
Z · a = h(m) · ρnH (b) · a
= h(m) · ρkH (a) · b
= h(m) · E,
because ρnH (b) · a = ρkH (a) · b, which follows from the same calculations as in Re-
mark 2.1.
Now let G be the semigroup of 3 × 3 matrices over the group ring Z7 [A5 ],
where A5 is the alternating group on 5 elements. The inner automorphism ρH is a
conjugation by a matrix H ∈ GL3 (Z7 [A5 ]). It is
ρH (L) = H −1 LH and ρrH (L) = H −r LH r
for any matrix L ∈ G and any r ∈ N, r > 0.
Remark 4.1. The semigroup of 3 × 3 matrices over the group ring Z7 [A5 ] is
used, because the multiplication can be calculate very efficient in this semigroup
and it provides a large key space (see [KKS13]).
Note that the element a has no inverse in G if M has no inverse in G.

A technique to obtain an invertible matrix H is presented in [HKKS13, Section 8].
From there it is also known that the exponents n and k should be of the magnitude
of 2t , where t is the security parameter, to make brute force search (for n and k)
infeasible.
Remark 4.2. Alice has to take care that H and HM do not commute.
Assume that H and HM commute, it is
Z = h(m) · H −n−k (HM )k H n

= h(m) · H −k (HM )k
= h(m) · b.
Hence, it adds up to calculate a new b if an eavesdropper, Eve, wants a new Z to

impersonate herself as Alice. This is discussed in the Section 4.1 under (I) (1).
The signature with G the semigroup of 3×3 matrices over the group ring Z7 [A5 ]
is summarized in Table 7.
Table 7. Signature with G the semigroup of 3 × 3 matrices over

the group ring Z7 [A5 ].
Public Parameters
G the semigroup of 3 × 3 matrices with entries in Z7 [A5 ], an invertible H ∈ G for
the automorphism ρH and a qualified hash function h.
Alice Bob
Choose n ∈ N and M ∈ G private.
Compute
(a, ρnH ) := (M, ρH )
n
with a := ρH (M ) · ρn−2
n−1
H (M ) · . . . · ρH (M ) · M
= H −n (HM )n .
Take care that a−1 ∈ G and that H and HM
do not commute.
Public Key: a
Choose message m and compute value h(m) ∈ G
Pick an ephemeral key k and compute
(b, ρkH ) := (M, ρH )k
k−1 k−2
with b := ρH (M ) · ρH (M ) · . . . · ρH (M ) · M
−k
= H (HM ) . k
Compute
Z := h(m) · ρn −n−k (HM )k H n .
H (b) = h(m) · H
Signature: (k, b, Z, m)
Compute
(a, x) · (b, ρkH ) = (ρkH (a) · b, x · ρkH ),

=:E
it is E = H −(k+n) (HM )n+k .
Prove
Z · a = h(m) · H −n−k (HM )k+n
= h(m) · E.
4.1. Security of the signature. The eavesdropper, Eve, knows Alice’s public
key a = H −n (HM )n . Eve wants to impersonate herself as Alice, i.e., everyone
should think that Eve’s new message m comes from Alice. Assume that Eve
knows the signature S = (k, b, Z, m).
(I) Eve chooses a new key k :

She chooses new parameters (k , b , Z , m ) where m is the new message.
(1) She has to calculate a new b .
(a) She needs to know the element M ∈ G which is one of Alice’s secrets.
She can get M from
√
H −1 · H k · b = H −1 · k H k · H −k (HM )k
k

= H −1 · k (HM )k
= M.
The difficulty here is to take the k-th root from the element (HM )k .
This is a difficult problem in a finite semigroup of 3 × 3 matrices over
the group ring Z7 [A5 ].
If it was easy to calculate the correct k-th root from (HM )k , Eve

could calculate the element b = H −k (HM )k .
(b) Alternatively she uses a new k with the property k := k · s, with

s ∈ N, s > 1. Now it is, with b = H −k (HM )k ,

s s
u := H k · b = (HM )k
= (HM )k·s

= (HM )k

and it is b = H −k · u = H −k · (HM )k . To prevent this, Alice
and Bob could agree that Alice uses only prime numbers for the
ephemeral keys k. If Bob gets a signature with k not a prime
number he recognizes that Eve tried such an attack.
(c) Suppose, Eve knows several signatures
S1 = (k1 , bk1 , Zk1 , m1 ) ,
S2 = (k2 , bk2 , Zk2 , m2 ) ,
..
.
Su = (ku , bku , Zku , mu ) ,
with pairwise different ephemeral keys ki . She can use the element
bki = H −ki (HM )ki to get
Tki := H ki · bki = (HM )ki .
It is
Tki +kj = Tki · Tkj = (HM )ki +kj .
The new bki +kj is now
bki +kj = H −(ki +kj ) · Tki +kj = H −(ki +kj ) · (HM )ki +kj .
In general Eve can calculate every bk with

u
k = αi · ki with αi ∈ N ∪ {0}.
i=1
If it is claimed that Alice’s private key a has no inverse, then M can

not have an inverse; hence HM has no inverse. Therefore αi can not
be a negative number. Thus Eve can calculate bk whereby every new
k is always greater than the smallest number ki .
A possible counter-measure is that Alice chooses at each new sig-
nature a smaller new ephemeral key than she uses for the previous
signature. This leads to the problem, that Alice can just perform,
with her private key n, a finite number of signatures, which depend
on her first ephemeral key k1 .
(2) After she has a new b she needs a new element Z = h(m ) · ρnH (b ). There
are two possibilities:
(a) Eve tries to recover n from the public element a = H −n (HM )n . Note
that Eve only knows the element HM if she can take the k-the root
of the element (HM )k (see above (1) (a)).
As said in [HKKS13], a special case of this problem, where H = I,
is the discrete logarithm problem for matrices over Z7 [A5 ]. This
problem is hard; it is addressed in [KKS13] in more detail.
M. Habeeb, D. Kahrobaei, C. Koupparis and V. Shpilrain also an-
alyze whether or not any information about the private exponent
n is leaked from transmission, i.e., from the fact that Eve knows
a = H −n (HN )n . That is, for a random exponent n, how different is
the matrix in the first component of (M, ρH )n = (H −n (HM )n , ρnH )
from N , where N is a random matrix? They find out, that no in-
formation about a private exponent n is revealed from the public
element a = H −n (HM )n (see [HKKS13, Section 7]).
(b) She does not know the secret n, therefore she has to calculate Z in
another way. Eve knows that Bob will verify the signature by the
proof of the following equation

Z · a = h(m ) · ρkH (a) · b .
She can calculate Z as

Z = h(m ) · ρkH (a) · b · a−1
if the inverse of the element a exists.

Therefore, to prevent an attack (I) from Eve, Alice should assure that her public
element a has no inverse. Hence, she can create the signature only in a
semigroup. The element a = H −n (HM )n has no inverse if the matrix M is not
invertible.
(II) Eve uses the same key k:

Eve chooses a new message m . The elements k and b are the same. She only needs
a new element Z . Hence she calculates

Z = h(m ) · (h(m))−1 · Z
= h(m ) · (h(m))−1 · h(m)ρnH (b)
= h(m ) · ρnH (b).
Therefore it is very easy for Eve to make everyone believe that her message m
comes from Alice. Alice and Bob could take care that every ephemeral k is
used only once.
(III) Eve’s information from Z:
(1) Let us take a look at the situation if she wants to get the private key
n with the help from Z. Note that the hash function h is public. It is
Z = h(m) · ρnH (b) and it follows
A := H k · (h(m))−1 · Z = H k · ρnH (b)
= H k · H −n−k (HK)k H n
= H −n (HM )k H n .

:=B
Eve knows B from H k · b = H k · H −k (HM )k = B. Eve can get x := H n

if she solves the conjugations search problem, that is: given two con-
jugate element A, B ∈ G, find a particular element x ∈ G such that
x−1 Bx = A. Suppose that she solves this problem and gets H n , she then
has to solve the discrete logarithm problem for matrices over Z7 [A5 ],
namely recover n from H and H n . This problem is hard (see [KKS13]
for more details).
(2) Suppose Eve knows several signatures
S1 = (k1 , bk1 , Zk1 , m1 ) ,
S2 = (k2 , bk2 , Zk2 , m2 ) ,
..
.
Su = (ku , bku , Zku , mu ) ,
with pairwise different ephemeral keys ki .
With Zkj = h(mj ) · H −n−kj (HM )kj H n follows
Xkj := H kj · (h(mj ))−1 Zkj = H −n (HM )kj H n .
With very similar deliberations as in (I) (1) (b) and (c) we have:
(a) Eve chooses a new k with the property k := ki · s, with s ∈ N, s > 1.
It is
s
Xk =ki ·s := (Xki ) = H −n (HM )ki H n
s
= H −n (HM )ki ·s H n

= H −n (HM )k H n .
For this k she can get Zk , for the signature (k , bk , Zk , m ), with

Zk = h(m ) · H −k · Xk

= h(m ) · H −k −n (HM )k H n .
To prevent this, Alice and Bob could agree that Alice uses only prime
numbers for the ephemeral keys k. If Bob gets a signature with
k not a prime number he recognizes that Eve tried such an attack.
(b) As above in (a) it is
Xkj := H kj · (h(mj ))−1 Zkj = H −n (HM )kj H n .
It follows
Xkj +ki := Xkj · Xki = H −n (HM )kj +ki H n .
The new Zkj +ki for the signature (kj + ki , bkj +ki , Zkj +ki , m ) is now
Zkj +ki = h(m ) · H −(kj +ki ) · Xkj +ki
= h(m ) · H −(kj +ki )−n (HM )kj +ki H n .
In general Eve can calculate every Zk with

u
k = αi · ki with αi ∈ N ∪ {0}.
i=1
If it is claimed that the private key a from Alice has no inverse, then
M can not have an inverse; hence HM has no inverse. Therefore αi
can not be a negative number. Thus Eve can calculate Zk whereby
every new k is always greater than the smallest number ki .
A possible counter-measure is that Alice chooses at each new sig-
nature a smaller new ephemeral key than she uses for the previous
signature. This leads to the problem, that with her private key n,
Alice can only perform a finite number of signatures depending on
her first ephemeral key k1 .
If Eve tries to impersonate herself as Alice with the information from (III) she also
needs the corresponding bk , which is discussed in (I) (1).
The counter-measure from Alice against Eve’s attacks (II) and (III) should be,
to determine, that the image of the hash function h is only the non-invertible
matrices from the semigroup G. Hence Eve dose not know the element ρnH (bkj )
and therefore she can not use Xkj for an attack.
Note added in proof

In the paper [MKU14] the authors introduce an embedding of Mat3 (F7 [A5 ])
into Mat180 (F7 ) and a linear algebra attack to break the particular instance of the
protocol from [HKKS13]. This also affects the example in Section 3.3 and the
security analysis (I)(2)(b) in Section 4.1.
References
[DH79] W. Diffie and M. E. Hellman, New directions in cryptography, IEEE Trans. Information
Theory IT-22 (1976), no. 6, 644–654. MR0437208 (55 #10141)
[ElG85] T. ElGamal, A public key cryptosystem and a signature scheme based on dis-
crete logarithms, IEEE Trans. Inform. Theory 31 (1985), no. 4, 469–472, DOI
10.1109/TIT.1985.1057074. MR798552 (86j:94045)
[HKKS13] M. Habeeb, D. Kahrobaei, C. Koupparis and V. Shpilrain, Public key exchange using
semidirect product of (semi)groups, in: ACNS 2013, Lecture Notes Comp. Sc. 7954,
(2013), 475-486.
[KK06] D. Kahrobaei and B. Khan, A non-commutative generalization of ElGamal key ex-
change using polycyclic groups, Proceeding of IEEE, GLOBECOM (2006), 1-5.
[KK12] D. Kahrobaei and C. Koupparis, Non-commutative digital signatures, Groups Com-

plex. Cryptol. 4 (2012), no. 2, 377–384. MR3043439
[KKS13] D. Kahrobaei, C. Koupparis, and V. Shpilrain, Public key exchange using matrices
over group rings, Groups Complex. Cryptol. 5 (2013), no. 1, 97–115. MR3065451
[MKU14] A.D. Myasnikov, M. Kreuzer and A. Ushakov, A linear algebra attack to group-ring-
based key exchange protocols, I. Bourenau, P. Owesarski and S. Vaudenay (eds.), ACNS
2014, LNCS 8479, pp. 37–43, Springer International International Publishing, Switzer-
land, 2014.
[MSU08] A. Myasnikov, V. Shpilrain, and A. Ushakov, Group-based cryptography, Advanced
Courses in Mathematics. CRM Barcelona, Birkhäuser Verlag, Basel, 2008. MR2437984
(2009d:94098)
[MvOV97] A. J. Menezes, P. C. van Oorschot, and S. A. Vanstone, Handbook of applied cryptog-
raphy, CRC Press Series on Discrete Mathematics and its Applications, CRC Press,
Boca Raton, FL, 1997. With a foreword by Ronald L. Rivest. MR1412797 (99g:94015)
[MW97] A. J. Menezes and Y.-H. Wu, The discrete logarithm problem in GL(n, q), Ars Combin.
47 (1997), 23–32. MR1487162 (98j:11122)
Fachbereich Mathematik, Universität Hamburg, Bundesstrasse 55, 20146 Hamburg,

Germany.
E-mail address: anja.moldenhauer@uni-hamburg.de
Volume 633, 2015
On some algorithmic properties of finite state

automorphisms of rooted trees
Benjamin Steinberg
Abstract. We survey some algorithmic properties of finite state automor-
phisms of a regular rooted tree. These results have been independently ob-
served by the author and others, but they have never been published. We also
list some open problems.
1. Introduction and main results

The purpose of this contribution is to survey some algorithmic properties of
automorphisms of regular rooted trees given by finite initial automata. A number
of useful results have been independently observed by the author and others, but
have never been published to the best of my knowledge.
The first part of the article offers a partial solution to a question of Grigorchuk,
Nekrashevych, Sushchanskii and Šuniḱ [5,6]. I had placed this part of the article on
the ArXiv in July 2006 and was informed by Miklos Ábert that he too had obtained
this partial solution. Neither of us published the result because we wanted to obtain
the full solution. Given that a full solution has not yet been achieved, it now seems
worthwhile to place the partial results in the literature.
Let Tk be the rooted regular k-ary tree. We view it as the Cayley graph of
the free monoid A∗k , where Ak = {0, . . . , k − 1} is the standard alphabet of size k.
In particular, we identify vertices with words. It is well known that Aut(Tk ) is a
profinite group. In fact, denoting by Sk the symmetric group acting on the left of
Ak , there is a permutational wreath product decomposition
(1) (Aut(Tk ), Tk ) = (Sk , Ak ) (Aut(Tk ), Tk ) = (Sk Aut(Tk )k , Ak × Tk )
(cf. [1, 2, 5]) and hence
n times

Aut(Tk ) = (Sk , Ak ) (Sk , Ak ) · · · = lim (Sk , Ak ) · · · (Sk , Ak ).
←−
n∈N
For more on this group see [1, 2, 5, 6, 9]. An element f ∈ Aut(Tk ) is said to be
spherically transitive if, for each n, f
acts transitively on the set of vertices at
2010 Mathematics Subject Classification. Primary 20F10.

Key words and phrases. Automata, spherical transitivity, iterated wreath products, rooted
trees, rational power series, word problem, linear space.
This work was partially supported by a grant from the Simons Foundation (#245268 to
Benjamin Steinberg) and by an NSERC grant.
2015
115
116 BENJAMIN STEINBERG
distance n from the root, i.e., transitively on the set of words of length n [1,2,5,6,9].
This is equivalent to topological transitivity and ergodicity of the action on the
boundary ∂Tk [5].
If f ∈ Aut(Tk ) has wreath product decomposition
f = λf (f |0 , . . . , f |k−1 )
as per (1), then f |i is called the section of f at i ∈ Ak . (The notation is intended
to be suggestive of restricting f to the ith -subtree hanging from the root.) The
notation λf shall be used throughout for the element of Sk associated to f . One
can then define inductively, for any word w ∈ A∗k , the section f |w by the formula
f |ua = (f |u )|a where a ∈ Ak and u ∈ A∗k . Of course, f |ε = f , where ε is the empty
word. One then has the formula f (uw) = f (u)f |u (w) for any words u, w ∈ A∗k . An
element f ∈ Aut(Tk ) is said to be finite state if it has only finitely many distinct
sections. This is equivalent to saying that f can be computed by a finite state
automaton.
A finite state automaton over an alphabet A is a 4-tuple A = (Q, A, δ, λ) where
Q is a finite set of states, δ : Q×A → Q is the transition function and λ : Q×A → A
is the output function. We set q|a = δ(q, a) and q(a) = λ(q, a) for q ∈ Q, a ∈ A.
We extend this to words by the formulas:
(2) q|au = (q|a )|u ,
(3) q(au) = q(a)q|a (u).
So each state q ∈ A gives rise to a function A∗ → A∗ (in fact an endomorphism of
the rooted Cayley tree of A∗ ), via (3), which we also denote by q. An automaton
with a distinguished state is called an initial automaton.
Automata are usually represented by Moore diagrams. The Moore diagram for
A is a directed graph with vertex set Q. The edges are of the form
a|q(a)
q −−−−→ q|a .
Figure 1 gives the Moore diagram for a certain two-state automaton studied by
Grigorchuk and Żuk [7].
1|1
0|0 a b 1|0
0|1
Figure 1. Moore diagram for the lamplighter automaton
It is sometimes convenient to define, for q ∈ Q, the state function λq : A → A

given by
λq (a) = q(a) = λ(q, a).
If, for each q ∈ Q, the state function λq is a permutation, that is, belongs to the
symmetric group SA on A, then one can easily verify that each state q computes
a permutation of A∗ [5, 9]. We call such an automaton invertible. In particular, if
ALGORITHMIC PROPERTIES OF FINITE STATE AUTOMORPHISMS 117
the alphabet of the invertible automaton is Ak and q is a state, then the function
q belongs to Aut(Tk ) = Sk Aut(Tk ). The wreath product coordinates of q are:
(4) q = λq (q|0 , . . . , q|k−1 )
and so our two uses of the notations λq and q|i are consistent.
For example, the automaton from Figure 1 is described in wreath product
coordinates by a = (a, b), b = (01)(a, b). More generally, if w ∈ A∗k , then the
section of q at w is exactly the state q|w and in particular the transformation q
is finite state. One can show [5, 9] that the inverse of q is given by the finite
state automaton obtained by switching the two sides of the labels of the Moore
diagram and choosing as the initial state the state corresponding to q. If A is
an invertible automaton, then G(A) denotes the group generated by the states of
A. Such groups are called automaton groups and constitute the main examples of
finitely generated self-similar groups [9]. For instance the group generated by the
states of the automaton in Figure 1 is the lamplighter group Z Z/2ZZ [5, 7, 12].
If f ∈ Aut(Tk ) is finite state, then it can be computed by the initial automaton
whose state set is Q = {f |w : w ∈ A∗ } (note: this set is finite by assumption).
The transition and output functions are given by δ(f |w , a) = f |wa and λ(f |w , a) =
f |w (a). The initial state is f |ε = f . We remark that the composition of finite state
transformations is also finite state [3, 5, 9] and so the collection of invertible finite
state automorphisms is a subgroup of Aut(Tk ).
If H is a profinite group, we denote by [H, H] the closure of the commutator
subgroup of H. The abelianization H/[H, H] of H shall be denoted H ab and is
again a profinite group.
Let (G, Ak ) be a transitive permutation group. Then the infinite permutational
wreath product
(5) G = ∞ (G, Ak ) = (G, Ak ) (G, Ak ) · · ·
is a closed subgroup of Aut(Tk ). Moreover, it acts spherically transitively on Tk [2].
The abelianization G is well known to be isomorphic to the infinite
ab of G
direct product G × G × · · · [2, Chapter 4, Proposition 4.3]. To describe the
ab ab
map, we think about G ab in a different way. Since Gab is a finite abelian group,
it is a finite direct product of cyclic groups of prime power order in an essentially
unique way. Hence we can view it as the additive group of a finite commutative
ring via this decomposition. In particular, if Gab is cyclic of prime order p, we view
it as the additive group of the field of p elements. We can then identify G ab with
the additive group of the ring of formal power series G t over G in a single
ab ab
variable t. If s ∈ Gab t, we use the notation s, tn

to denote the coefficient of tn
in s. The abelianization map, with this notation, is given by:

(6) G],
g[G, tn
= λg|w [G, G];
|w|=n
see [2].
The importance of the abelianization map is reflected in the following theo-
rem [2, Chapter 4, Propositions (4.6) and (4.7)].
Theorem 1 ([2]). Let G = ∞ (Z/kZ, Ak ). Then:
is spherically transitive if and only if its abelianization
(1) an element g ∈ G
G]
g[G, ∈ Z/kZt satisfies g[G,
G],
tn
∈ Z/kZ× , for all n ≥ 0;
are conjugate if and only if

(2) two spherically transitive elements f, g ∈ G

they have the same image in G = Z/kZt.
ab
We sketch a proof of the first part of the theorem. The proof goes by induction
on the level of the tree and we merely illustrate how the inductive step works. The
key point is that g
acts transitively on Ank if and only if it acts transitively on
An−1
k and, for each word u ∈ An−1 k , the stabilizer of u in g
acts transitively on
n−1
uAk . Now if we assume that g acts as a kn−1 -cycle σ on An−1 k , then g k generates
the stabilizer in g
of every word in Ak . Let us reorder the elements of Ank so
n−1
that Ank = {w1 , . . . , wkn } and σ(wi ) = wi+1 (with indices identified modulo kn ).
Using this ordering of the elements Ank , we can write g = σ(g|w1 , . . . , g|wkn ) in the
n
semidirect product decomposition Aut(Tk ) = SAnk Aut(Tk )Ak . A straightforward
n−1
calculation then shows that g k = (h1 , . . . , hkn ) where
hi = g|wi−1 g|wi−2 · · · g|w1 g|wkn g|wkn −1 · · · g|wi .
G],
tn
, for all i. It follows that g kn−1 acts
In particular, λhi = |w|=n λg|w = g[G,
transitively on uAk for all u ∈ An−1 if and only if g[G, G],
tn
∈ Z/kZ× .
k
Let us return to the setting where (G, Ak ) is a transitive permutation group
and let G be as in (5). It is easy to see from (4) that if A = (Q, Ak , δ, λ) is a finite
state automaton, then G(A) ≤ G if and only if λq ∈ G for all q ∈ Q.
We are now in a position to present the results that will be proved in the first
part of the paper. Again, I recall that these were obtained independently by Miklos
Ábert (unpublished) and the author in 2006.
Theorem 2. Let g ∈ ∞ (Z/kZ, Ak ) be a finite state transformation given by a
finite state initial automaton. Then it is decidable whether f is spherically transi-
tive.
The following corollary was pointed out to me by Zoran Šuniḱ.
Corollary 3. Let g1 , . . . , gn ∈ ∞ (Z/kZ, Ak ) be finite state elements given by
finite state initial automata. Then it is decidable whether the group generated by
g1 , . . . , gn contains a spherically transitive element, and if so one can produce such
an element.
The argument is that the image of G = g1 , . . . , gn
in the abelianization
Z/kZt is finite and in fact consists of the cosets of the elements in the set
X = {g1m1 . . . gnmn : 0 ≤ mi ≤ k}. So, by Theorem 1, it follows that G con-
tains a spherically transitive element if and only if X does. This can be tested by
Theorem 2 and an explicit example can be produced if one exists.
Our next theorem concerns conjugacy of spherically transitive finite state au-
tomorphisms.
Theorem 4. Let f, g ∈ G = ∞ (Z/kZ, Ak ) be spherically transitive finite state
automorphisms given by finite state initial automata. Then it is decidable whether

f and g are conjugate in G.
Theorem 4 can be deduced from Theorem 1 and the following theorem.
Theorem 5. Let (G, Ak ) be a transitive permutation group and let G =
∞
(G, Ak ). Let f, g ∈ G be finite state transformations, given by finite state initial
automata. Then it is decidable whether f and g are equal in G ab .
The key idea for proving these results was inspired by Schützenberger’s theory
of automata and rational power series [10, 11]. In fact, a biproduct of the proofs
is:
Theorem 6. Let (G, Ak ) be a transitive permutation group and let G =
∞
(G, Ak ). Let f ∈ G be a finite state transformation. Then f [G,
G]
∈ Gab t
is a rational power series.
The second part of the paper is dedicated to proving the following result, which
has been observed independently by the author and several computer scientists, but
does not appear to be widely known to mathematicians. The reader is referred to [8]
for basics on formal languages and time/space complexity.
Theorem 7. Let A1 , . . . , An be a collection of invertible finite state initial au-
tomata. Then the word problem for the group G generated by these automata can be
solved in non-deterministic linear space. Equivalently, the language of words repre-
senting the trivial element of G is context-sensitive. In particular, each automaton
group has a context-sensitive word problem.
Non-deterministic linear space can be simulated in exponential time, but is
generally believed to be a proper subclass of exponential time.
The final section of the paper lists some open problems.
2. Spherical transitivity
If A is an initial automaton with state set {1, . . . , n}, then the adjacency matrix
A of A is given by putting Aij to be the number of directed edges from state i to
state j. The results concerning spherical transitivity and the abelianization of finite
state automorphisms all rely on the following observation.
Lemma 8. Let (G, Ak ) be a transitive permutation group and let G be as in
be computed by an automaton A with state set {1, . . . , n} and initial
(5). Let g ∈ G
state 1. Let A be the adjacency matrix of A and let vA be the vector whose entries
are given by (vA )i = λi [G, G], i = 1, . . . , n. Then
∞

G]
g[G, = (Aj vA )1 tj .
j=0
Proof. It is well known that (A )rs counts the number of paths in A of length
j
j from r to s. Thus (Aj vA )1 sums over all paths p of length j from the initial state
1 the value of vA at the endpoint of p. That is, we have

(Aj vA )1 = λ1|w [G, G] = G],
λg|w [G, G] = g[G, tj
|w|=j |w|=j
where the last equality follows from (6).
Proof of Theorem 2. By Theorem 1, the automorphism g is spherically tran-

G]
sitive if and only if each coefficient of g[G, belongs to Z/kZ× . By Lemma 8,
we thus want to check whether (keeping the above notation) (Aj vA )1 ∈ Z/kZ× for
each j ≥ 0. Since (Z/kZ)n has kn elements, Ar vA = As vA for some 0 ≤ r < s ≤ kn
and so the above condition is a finite check.
Proof of Theorem 5. Let (G, Ak ) be a transitive permutation group and let G

be as in (5). Let A and B be initial automata computing f and g, respectively. Say
that A has m states and B has n states. Let A and B be the respective adjacency
matrices of A and B. Let vA and vB be the associated vectors, as per Lemma 8.
Consider the matrix !
A 0
M= .
0 B
. . . , em+n } be the standard basis of row vectors for (Gab )m+n and set
Let {e1 , !
vA
v= . Then, applying Lemma 8, we have for j ≥ 0:
vB
G],
(e1 − em+1 )(M j v) = (Aj vA )1 − (B j vB )1 = f [G, tj
− g[G,
G],
tj
.
Hence f [G, G] = g[G,

G]
if and only if (e1 − em+1 )(M j v) = 0 for all j ≥ 0. But
again, M v = M v some 0 ≤ r < s ≤ |Gab |m+n , so we can check this.
r s
If Gab is a finite field, then we can do better. Indeed, since the vectors
v, M v, . . . , M m+n v in (Gab )m+n must be linearly dependent, it follows that for
some 0 ≤ i ≤ m + n, M i v = c0 v + c1 M v · · · + ci−1 M i−1 v. Such a recursion im-
plies that M j v is a linear combination of v, M v, . . . , M n+m−1 v for all j ≥ n + m.
Hence (e1 − em+1 )(M j v) = 0 for all j ≥ 0 if and only if (e1 − em+1 )(M j v) = 0 for
0 ≤ j ≤ m + n − 1.
Remark 9. The proof of Theorem 5 allows for an alternative algorithm for
testing spherical transitivity for Aut(T2 ). By Theorem ∞1, g ∈ Aut(T2 ) is spheri-
cally transitive if and only if g[Aut(T2 ), Aut(T2 )] = n=0 tn , and all spherically
transitive elements are conjugate. The so-called odometer a = (01)(1, a) is one
such spherically transitive element and it has two distinct sections, that is, it can
be computed by a two-state automaton. It follows from the proof of Theorem 5
that if g ∈ Aut(T2 ) is computed by an n-state initial automaton with adjacency
matrix A, then one needs only to verify (Aj vA )1 = 0 for 0 ≤ j ≤ n + 1. This special
case has been folklore for quite some time.
Proof of Theorem 6. From Lemma 8 that we have g[G, G]
= ((I − At)−1 vA )1 .
Since
1
(I − At)−1 = Adj(I − At)
det(I − At)
and each entry of the classical adjoint Adj(I − At) is a polynomial in t, as is
det(I − At), it follows that the entries of (I − At)−1 are rational power series in t.
Since ((I − At)−1 vA )1 is a linear combination of entries of (I − At)−1 , it follows
G]
that g[G, is a rational power series.
3. Space complexity of the word problem

Let S = {A1 , . . . , Ak } be a collection of invertible initial automata over an
alphabet A with respective states sets Q1 , . . . , Qk . Without loss of generality we
may assume that S is a symmetric generating set. The point of this section is
to observe that the word problem for the group G = A1 , . . . , Ak
is decidable
in non-deterministic linear space (and hence the word problem for G, viewed as a
language, is context sensitive). This fact is essentially known to computer scientists
(although I may have been the first to observe it in this context), but does not seem
to be widely known to group theorists so we record it here. Let me first recall the
definition of non-deterministic linear space.
A linear bounded automaton (LBA) is a halting, non-deterministic Turing ma-
chine M with an input tape (which holds the input word and is never written on)
and some fixed number c of work tapes. What distinguishes and LBA from a Tur-
ing machine is that the LBA can only use as many cells of each work tape as the
length of the input word. Hence if the input word w has length n, then M can
use at most cn cells in any branch of its computation on w. A language L is in
non-deterministic linear space if there is an LBA accepting it. This is equivalent
to L being generated by a context-sensitive grammar. Note that non-deterministic
linear space is closed under complementation. Details on language theory can be
found in [8].
An LBA can be simulated by a deterministic Turing machine in exponential
time (because there are only exponentially many configurations the LBA can ever
be in during any branch of computation for a given input word). However, it is
generally believed that the class of polynomial-space languages is properly contained
in the class of exponential-time languages.
We recall here the construction of an initial automaton A computing the prod-
uct Ai1 · · · Ain . The state set is Qi1 × · · · × Qin . The transitions are of the form
a|q1 ···qn (a)

(7) (q1 , . . . , qn ) −−−−−−−−→ (q1 |q2 ···qn (a) , . . . , qn−1 |qn (a) , qn |a )
and the initial state is the n-tuple of initial states of Ai1 , . . . , Ain .
A map f computed by an initial automaton B with r states is not the identity
mapping if and only if there is a state q reachable from the initial state with λq not
the identity. Such a state can be reached from the initial state by a path of length
at most r − 1. If w labels this path and if λq (a) = a, then f (wa) = f (w)q(a) = wa.
Thus if f is not the identity, then it acts non-trivially on a word of length at most
r where r is the number of states of B.
Let m = max{|Q1 |, . . . , |Qk |}. Then A has at most mn states. We now describe
an LBA M with 2 work tapes that solves the co-word problem for G, i.e., decides
if a word in the generators does not represent the identity. Given an input word
Ai1 · · · Ain in the generators, it tries to non-deterministically guess a word of length
at most mn on which the automaton A described above acts non-trivially. The first
work tape stores the current state of A (which occupies n-cells because it is an n-
tuple). The second work tape stores the number of letters we have already guessed,
written in base m + 1; since we will only check words of length up to mn , we can
store this also in n-cells.
The machine initially has the the initial state of A on the first work tape and 0
on the second work tape. At each step of a branch of computation, M first checks
if the second work tape contains mn ; if so M halts this branch of the computation
as unsuccessful. Otherwise, M non-deterministically guesses an input letter a,
augments the number in the second work tape by 1 and then proceeds as follows.
Say that the first work tape is in the state (q1 , . . . , qn ) of A. Then M scans the
the first work tape from right to left successively computing qi · · · qn (a) and writing
qi |qi+1 ···qn (a) on top of qi , i.e., M simulates the transition (7). If q1 · · · qn (a) = a,
then M halts accepting Ai1 · · · Ain as not being the identity; otherwise M repeats
the above steps.
Since A has at most mn states, the discussion above shows that Ai1 · · · Ain is
not the identity if and only if it acts non-trivially on a word of length at most mn .
But M non-deterministically simulates A on all inputs of length at most mn and so
M will correctly determine whether Ai1 · · · Ain is non-trivial. We have thus proved
Theorem 7.
4. Open problems
There are a number of open problems remaining. Many of these are already
in [5].
Question 1. Is there an algorithm to determine if an invertible initial automa-
ton is spherically transitive?
The answer is positive for binary trees (this is a folklore result, but also follows
from the results above).
Question 2. Is there an algorithm to determine if two invertible initial au-
tomata over Ak are conjugate in Aut(Tk )?
It is known that all spherically transitive automorphisms are conjugate to the
odometer [5] so a positive answer to Question 2 implies a positive answer to Ques-
tion 1.
Question 3. Is there an algorithm to determine if an invertible initial automa-
ton has infinite order?
It has recently been shown that the answer to Question 3 is “no” if we allow
non-invertible automata [4].
Question 4. Is there an algorithm to determine if a group generated by initial
automata (respectively, an automaton group) is finite?
Recently it was shown that finiteness is undecidable for automaton semigroups
[4].
Question 5. Does there exist a group generated by a finite number of initial
automata whose word problem is PSPACE-complete?
We strongly suspect the answer to Question 5 is “yes.” Note that if A1 , . . . , An
are initial automata generating a group with a PSPACE-complete word problem
and A is the disjoint union of these automata (with no initial state), then the
automaton group generated by A contains A1 , . . . , An
and so has a PSPACE-
complete word problem, as well. Thus Question 5 is equivalent to asking whether
there is an automaton group with a PSPACE-complete word problem.
References
[1] Laurent Bartholdi, Rostislav I. Grigorchuk, and Zoran Šuniḱ, Branch groups, Handbook
of algebra, Vol. 3, North-Holland, Amsterdam, 2003, pp. 989–1112, DOI 10.1016/S1570-
7954(03)80078-5. MR2035113 (2005f:20046)
[2] Hyman Bass, Maria Victoria Otero-Espinar, Daniel Rockmore, and Charles Tresser, Cyclic
renormalization and automorphism groups of rooted trees, Lecture Notes in Mathematics,
vol. 1621, Springer-Verlag, Berlin, 1996. MR1392694 (97k:58058)
[3] S. Eilenberg, Automata, Languages and Machines, Academic Press, New York, Vol. A, 1974;
Vol. B, 1976.
[4] Pierre Gillibert, The finiteness problem for automaton semigroups is undecidable, Internat.
J. Algebra Comput. 24 (2014), no. 1, 1–9, DOI 10.1142/S0218196714500015. MR3189662
[5] R. I. Grigorchuk, V. V. Nekrashevich, and V. I. Sushchanskiı̆, Automata, dynamical systems,
and groups (Russian, with Russian summary), Tr. Mat. Inst. Steklova 231 (2000), no. Din.
Sist., Avtom. i Beskon. Gruppy, 134–214; English transl., Proc. Steklov Inst. Math. 4 (231)
(2000), 128–203. MR1841755 (2002m:37016)
[6] R. I. Grigorchik and Z. Šuniḱ, On self-similarity and branching in group theory, to appear
in London Mathematical Society Lecture Note Series.
[7] Rostislav I. Grigorchuk and Andrzej Żuk, The lamplighter group as a group generated by
a 2-state automaton, and its spectrum, Geom. Dedicata 87 (2001), no. 1-3, 209–244, DOI
10.1023/A:1012061801279. MR1866850 (2002j:60009)
[8] John E. Hopcroft and Jeffrey D. Ullman, Introduction to automata theory, languages, and
computation, Addison-Wesley Publishing Co., Reading, Mass., 1979. Addison-Wesley Series
in Computer Science. MR645539 (83j:68002)
[9] Volodymyr Nekrashevych, Self-similar groups, Mathematical Surveys and Monographs,
vol. 117, American Mathematical Society, Providence, RI, 2005. MR2162164 (2006e:20047)
[10] M. P. Schützenberger, On the definition of a family of automata, Information and Control 4
(1961), 245–270. MR0135680 (24 #B1725)
[11] M. P. Schützenberger, On a theorem of R. Jungen, Proc. Amer. Math. Soc. 13 (1962), 885–
890. MR0142781 (26 #350)
[12] P. V. Silva and B. Steinberg, On a class of automata groups generalizing lamp-
lighter groups, Internat. J. Algebra Comput. 15 (2005), no. 5-6, 1213–1234, DOI
10.1142/S0218196705002761. MR2197829 (2007b:20072)
Department of Mathematics, City College of New York

E-mail address: bsteinberg@ccny.cuny.edu
Selected Published Titles in This Series
633 Delaram Kahrobaei and Vladimir Shpilrain, Editors, Algorithmic Problems of
Group Theory, Their Complexity, and Applications to Cryptography, 2015
632 Gohar Kyureghyan, Gary L. Mullen, and Alexander Pott, Editors, Topics in
Finite Fields, 2015
631 Siddhartha Bhattacharya, Tarun Das, Anish Ghosh, and Riddhi Shah, Editors,
Recent Trends in Ergodic Theory and Dynamical Systems, 2015
630 Pierre Albin, Dmitry Jakobson, and Frédéric Rochon, Editors, Geometric and
Spectral Analysis, 2014
629 Milagros Izquierdo, S. Allen Broughton, Antonio F. Costa, and Rubı́ E.
Rodrı́guez, Editors, Riemann and Klein Surfaces, Automorphisms, Symmetries and
Moduli Spaces, 2014
628 Anita T. Layton and Sarah D. Olson, Editors, Biological Fluid Dynamics:
Modeling, Computations, and Applications, 2014
627 Krishnaswami Alladi, Frank Garvan, and Ae Ja Yee, Editors, Ramanujan 125,
2014
626 Veronika Furst, Keri A. Kornelson, and Eric S. Weber, Editors, Operator
Methods in Wavelets, Tilings, and Frames, 2014
625 Alexander Barg and Oleg R. Musin, Editors, Discrete Geometry and Algebraic
Combinatorics, 2014
624 Karl-Dieter Crisman and Michael A. Jones, Editors, The Mathematics of
Decisions, Elections, and Games, 2014
623 Pramod N. Achar, Dijana Jakelić, Kailash C. Misra, and Milen Yakimov,
Editors, Recent Advances in Representation Theory, Quantum Groups, Algebraic
Geometry, and Related Topics, 2014
622 S. Ejaz Ahmed, Editor, Perspectives on Big Data Analysis, 2014
621 Ludmil Katzarkov, Ernesto Lupercio, and Francisco J. Turrubiates, Editors,
The Influence of Solomon Lefschetz in Geometry and Topology, 2014
620 Ulrike Tillmann, Søren Galatius, and Dev Sinha, Editors, Algebraic Topology:
Applications and New Directions, 2014
619 Gershon Wolansky and Alexander J. Zaslavski, Editors, Variational and Optimal
Control Problems on Unbounded Domains, 2014
618 Abba B. Gumel, Editor, Mathematics of Continuous and Discrete Dynamical Systems,
2014
617 Christian Ausoni, Kathryn Hess, Brenda Johnson, Wolfgang Lück, and Jérôme
Scherer, Editors, An Alpine Expedition through Algebraic Topology, 2014
616 G. L. Litvinov and S. N. Sergeev, Editors, Tropical and Idempotent Mathematics
and Applications, 2014
615 Plamen Stefanov, András Vasy, and Maciej Zworski, Editors, Inverse Problems
and Applications, 2014
614 James W. Cogdell, Freydoon Shahidi, and David Soudry, Editors, Automorphic
Forms and Related Geometry, 2014
613 Stephan Stolz, Editor, Topology and Field Theories, 2014
612 Patricio Cifuentes, José Garcı́a-Cuerva, Gustavo Garrigós, Eugenio Hernández,
José Marı́a Martell, Javier Parcet, Keith M. Rogers, Alberto Ruiz, Fernando
Soria, and Ana Vargas, Editors, Harmonic Analysis and Partial Differential
Equations, 2014
611 Robert Fitzgerald Morse, Daniela Nikolova-Popova, and Sarah Witherspoon,
Editors, Group Theory, Combinatorics, and Computing, 2014
For a complete list of titles in this series, visit the

AMS Bookstore at www.ams.org/bookstore/conmseries/.
633 CONM
Group Theory, Complexity, Cryptography • Kahrobaei et al., Editors
This volume contains the proceedings of the AMS Special Session on Algorithmic Prob-
lems of Group Theory and Their Complexity, held January 9–10, 2013 in San Diego, CA
and the AMS Special Session on Algorithmic Problems of Group Theory and Applications
to Information Security, held April 6–7, 2013 at Boston College, Chestnut Hill, MA.
Over the past few years the field of group-based cryptography has attracted attention
from both group theorists and cryptographers. The new techniques inspired by algorithmic
problems in non-commutative group theory and their complexity have offered promising
ideas for developing new cryptographic protocols. The papers in this volume cover algo-
rithmic group theory and applications to cryptography.
ISBN 978-0-8218-9859-8
9 780821 898598
CONM/633
AMS

Algorithmic Problems of Group Theory, Their Complexity, and Cryptography

Enviado por

Dados do documento

Título original

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

Algorithmic Problems of Group Theory, Their Complexity, and Cryptography

Enviado por

Direitos autorais:

Formatos disponíveis

633

Algorithmic Problems of Group

Algorithmic Problems of Group Theory and Applications

American Mathematical Society

Algorithmic Problems of Group Theory and Applications

Algorithmic Problems of Group

Algorithmic Problems of Group Theory and Applications

American Mathematical Society

2010 Mathematics Subject Classiﬁcation. Primary 20-XX, 68-XX.

Library of Congress Cataloging-in-Publication Data

This volume consists of contributions by participants and speakers in special

Secret sharing using non-commutative groups

Bren Cavallo and Delaram Kahrobaei

3. Shamir’s Secret Sharing Scheme

4. Secret Sharing Using Non-commutative Groups

a set of words R ⊂ F (X) we deﬁne R

as the smallest normal subgroup of F (X)

. We call R the set of

4.3. Small Cancellation Groups. In this section we introduce a candidate

4.4. Secret Sharing and the Shortlex Ordering. Let X = {x1 , · · · , xn }

Support. Delaram Kahrobaei was partially supported by the Oﬃce of Naval

CUNY Graduate Center, City University of New York

An algorithm that decides conjugacy

2010 Mathematics Subject Classiﬁcation. Primary 20E06; Secondary 20F10.

(ii) If g is conjugate to an element g in some factor, but not in a conjugate of H,

Proof. Let g, g ∈ G. We will use Theorem 2.1 as a blueprint to derive an

and, thus, g2 = h−1 −1

(h−1 p1 h)(h−1 p2 h) · · · (h−1 pr h) = g1 g2 · · · gr .

(∗) h = ui and u−i p1 ui = g1

in F for some positive integer i. By Proposition 2.2, we would need to have g1 ∈

The following illustrates a more routine algorithmic approach:

Department of Mathematics, Brooklyn College, City University of New York,

Classiﬁcation of automorphic conjugacy classes

Bobbe Cooper and Eric Rowland

Abstract. We associate a ﬁnite directed graph with each equivalence class

2010 Mathematics Subject Classiﬁcation. Primary 20E36, Secondary 68R15.

To determine whether a word w is minimal, by Whitehead’s theorem it suﬃces

By counting two-letter subwords of w we can determine whether the length of

2. The graph Γ(W )

We suppress brackets here to emphasize that we have ﬁxed a representative of each

(P2) a looped path

We have referred to the double-looped vertex as a degenerate double-edged

Theorem 2.3. Let W be a root class containing an alternating minimal word.

Again by Lemma 1.5, ({x}, y) being level on w is equivalent to (xy)w = (xy)w +

Lemma 2.4 already provides enough information to restrict the outdegrees of

Corollary 2.5. If w ∈ C2 is a minimal word that is not a root word, then

Proof. We have already established that by deﬁnition of Γ(W ) the outdegree

Lemma 3.1. Let W be a non-root class that contains no semi-alternating min-

Proof. By Lemma 2.4, every vertex in Γ(W ) has outdegree at most 1. On

We use the following result in the proof of Lemma 3.6.

where again φ = ({b}, a).

Proof of Theorem 1.1. If W is a root class whose minimal words have

Therefore let W be a non-root class whose minimal words have length n ≥ 9.

|V (Γ(W ))| ≤ mx (v) + 1 + mx (v)

Therefore it suﬃces to show that if xi appears in a minimal word w of length n

Finally, there are 21 words of length 3 to check:

Now we start with alternating words. We need several lemmas.

Lemma 4.2. Suppose w0 is an alternating minimal word and φ is a one-letter

Proof. Write φ = ({y}, x). Since φ(w0 ) is alternating, we have (yy)φ(w0 ) = 0,

Corollary 4.3. For w ∈ C2 ,

If π maps x → x, y → y or x → x, y → y, then consider (yxy)φ1 (w0 ) =

be the preﬁx of w0 of length k. Then w0 = u · σ(u) · σ 2 (u) · σ 3 (u) · · · σ −1

σφ2 (w0 ) = ({σ(y)}, σ(x))σ(w0 )

If π maps x → x, y → y or x → x, y → y, then consider (yxy)φ1 (w0 ) =