Probabilistic Boolean Networks as Models of Gene Regulatory Networks:

Inference, Simulation, Intervention

Ilya Shmulevich University of Texas M. D. Anderson Cancer Center

Outline of the Presentation

Biological motivation
l

Why study genetic networks? What questions do we want to answer?

Requirements of our models

l

Boolean Formalism Biological Implications Probabilistic Boolean Networks

l

Dynamics, Uncertainty, Graphical Models, Influence and Sensitivity of Genes, Perturbation, Intervention, Sensitivity Analysis

Biological Motivation
Were in the era of holistic biology. Massive amounts of biological data await interpretation:
this calls for formal modeling and computational methods; l it opens up a window on dynamical and functional characteristics (physiology) of an organism and disease progression.
l
3

Biological Motivation
Genes are not independent. They regulate each other and act collectively. This collective behavior can be observed using microarrays. Interest is shifting to temporal, genome-wide expression profiling.
l l

Such studies benefit from microarray technology. e.g. clustering, PCA, multidimensional scaling, network inference.

The interrelationships among genes constitute gene regulatory networks.

4

Genetic Network Models: Goals

Must incorporate rule-based dependencies between genes
l

Rule-based dependencies may constitute important biological information.

Must allow to systematically study global network dynamics

l

In particular, individual gene effects on long-run network behavior. Small sample size, noisy measurements, robustness

Must be able to cope with uncertainty

l

Must permit quantification of the relative influence and sensitivity of genes in their interactions with other genes
l

This allows us to focus on individual (groups of) genes.

5

Boolean Formalism
Studies give rise to qualitative phenomena, as observed by experimentalists. Studied systems exhibit multiple steady states and switchlike transitions between them. It is experimentally shown that such systems are robust to exact values of kinetic parameters of individual reactions. For practical approximation, gene regulatory networks have been treated with a Boolean formalism (i.e. ON/OFF).
6

Boolean Formalism
Boolean idealization enormously simplifies the modeling task. We want to study the collective regulatory behavior without specific quantitative details. Boolean networks qualitatively capture typical genetic behavior.

Example

Basic Structure of Boolean Networks

A B 1 means active/expressed 0 means inactive/unexpressed
Boolean function ABX 001 011 100 111

In this example, two genes (A and B) regulate gene X. In principle, any number of input genes are possible. Positive/negative feedback is also common (and necessary for homeostasis).

Dynamics of Boolean Networks

A B 0 1 C 1 D 0 E 1 F 0 Time

0 C

1 D

1 E

0 F

A B

At a given time point, all the genes form a genome-wide gene activity pattern (GAP) (binary string of length n ). Consider the state space formed by all possible GAPs.
10

State Space of Boolean Networks

Similar GAPs lie close together. There is an inherent directionality in the state space. Some states are attractors (or limit-cycle attractors). The system may alternate between several attractors. Other states are transient.

Picture generated using the program DDLab.

11

Implications for biology

[see Huang, J. Mol. Med., 77, 469-480, 1999 for more details on the next 5 slides] Equate cellular states with attractors. Many different stimuli can lead to the same cellular state (differentiation, growth, apoptosis). Thus, in real cells, these states correspond to attractors.
l

e.g. radiation, chemotherapy.

These attractor states are stable under minimal perturbations (this corresponds to flipping some bits in the GAP).
l l

most perturbations cause the network to flow back to the attractor. some genes are more important (master genes) and changing their activation can cause the system to transition to a different attractor.
12

Implications for biology

This stability is physiologically important it allows the cell to maintain its functional state within the tissue even under perturbations. Nevertheless, cells do switch states, e.g. from quiescence to growth, usually when certain genes are affected by extracellular signals. The cell translates such signals into specific alterations of genes/proteins.
l

cell surface receptors are wired to master switches and are good targets for manipulation.
13

Implications for biology

Hysteresis: a change in the systems state caused by a stimulus is not changed back after the stimulus is withdrawn.
l l

Network simulations support this kind of memory. It may also account for the fact that adaptive changes are often preserved through many cell division generations. Stability and hysteresis could explain inheritance of gene expressions (without physical fixation of information in DNA).
14

Tumorigenesis
Disturbance of the balance between attractors could be caused by mutations affecting the wiring or activation of important genes.
l l l

for example, stabilizing the growth state could lead to tumorigenesis. such mutations change the size of the basins of attraction. since the state space is finite, an increase of one basin of attraction leads to a decrease of another, say, differentiation.
15

Drug discovery
Most research has focused on the linear paradigm.
l

manipulation of individual molecular targets

Robustness of attractor states explains why single-gene perturbations have had little success on the macroscopic level. Because of hysteresis, the off genes might not be good targets for reversing pathological effects. We must rethink the functions of genes: to regulate the dynamics of attractors.
16

Drug discovery
The goal should be to push a tumor cell out of the growth attractor and into apoptosis or differentiation attractor.
l

to accomplish this, we have to intervene with specific lever points. How to identify them?

17

Probabilistic Boolean Networks (PBN)

Share the appealing rule-based properties of Boolean networks. Robust in the face of uncertainty. Dynamic behavior can be studied in the context of Markov Chains.
l

Boolean networks are just special cases. Explicitly represent probabilistic relationships between genes.

Close relationship to Bayesian networks

l

Allows quantification of influence of genes on other genes.

18

Boolean networks are inherently deterministic

Conceptually, the regularity of genetic function and interaction is not due to hardwired logical rules, but rather to the intrinsic self-organizing stability of the dynamical system. Additionally, we may want to model an open system with inputs (stimuli) that affect the dynamics of the network. From an empirical viewpoint, the assumption of only one logical rule per gene may lead to incorrect conclusions when inferring these rules from gene expression measurements, as the latter are typically noisy and the number of samples is small relative to the number of parameters to be inferred.
19

Basic structure of PBNs

f 1 (i) c1(i) f 2 (i) c2(i) x'i

x1 x2 x3

cl(i)(i) xn f l(i)(i)

If we have several good competing predictors (functions) for a given gene and each one has determinative power, dont put all our faith in one of them!

20

Dynamics
Dynamics of PBNs can be studied using Markov Chain theory. From the Boolean functions, we can compute
l l l
1 001 1 010 P2+P4 000 P3 1 100 P4 011 P2 101 P1+P3 P1+P3 P2+P4 111 1 110

transition probabilities stationary distribution steady-state distribution (if it exists)

1

P1

We can ask the question: In the long run, what is the probability that some given gene(s) will be ON/OFF?
21

Random Gene Perturbations

Genes can sometimes change value with a small probability p.
l

The genome is not a closed system genes can be activated/inhibited due to mutagens, heat stress, etc.

Perturbation vector g {0,1} Pr{g i = 1} = E [g i ] = p

22

Random Gene Perturbations

If no genes are perturbed, the standard network transition function will be used. Observation:
l

For p > 0, the Markov chain corresponding to the PBN is ergodic.

Thus, the steady-state distribution exists. l Convergence partially depends on p.
l
23

Transition Probabilities

24

Uncertainty: Relationship to Bayesian networks

Bayesian networks are graphical models that represent probabilistic relationships between variables.
l l l l

They explicitly represent dependencies and independencies between variables. They specify a probability distribution. Marriage between machine learning (rule-based systems) and uncertainty in AI. Naturally allow to select a model, from a set of competing models, that best explains the expression data.

25

PBNs and Bayesian Networks

Bayesian networks are inherently static, although dynamic generalizations have been proposed. However, the process of learning the model structure & parameters is intractable (NP-hard). Bayesian networks are not rule-based. PBNs retain the attractive properties of Bayesian networks (e.g. probabilistic dependencies, model selection), but are rulebased and inherently dynamic. The basic building blocks of Bayesian Networks (conditional probabilities) can be obtained from PBNs.
26

Influence of genes
Some genes are more equal than others (in determining the value of a target gene) In a Boolean function, some variables have greater determinative power on the output. Influence is defined in terms of the partial derivative of the Boolean function and the underlying joint probability distribution of the inputs (efficient spectral methods exist) PBNs naturally allow us to compute influences between (sets of) genes
l

genes with a high influence would make potentially good targets for intervention.
27

Example

f ( x1 x2 x3 ) = x1 + x2 x3

28

Influence

29

Influence and Sensitivity

We can easily define the influence of a gene on another (set of) gene(s), in the PBN framework. We can also define the sensitivity of a gene (definition omitted here).
l

Biologically, this represents the stability, or in some sense, the autonomy of a gene.

30

Long-term Influence
0.9 0.8

0.7

0.6

0.5

I2 (x3 )

0.4

0.3 I2 (x1 ) 0.2

I2 (x2 )

10

20

30

40

50 time-step

60

70

80

90

100

31

Intervention
One of the key goals of PBN modeling is the determination of possible intervention targets (genes) such that the network can be persuaded to transition into a desired state or set of states. Clearly, perturbation of certain genes is more likely to achieve the desired result than that of some other genes. Our goal, then, is to discover which genes are the best potential lever points in the sense of having the greatest possible impact on desired network behavior.
32

Example
Clearly, the choice in this simple example should be gene x1.

1 001 1 010 P2+P4 000 1 P3 1 100 P4 011 P2 101 P1+P3 P1+P3 P2+P4 111 1 110

P1

33

Intervention

The problem of intervention is posed as: reaching a desired state as early as possible. We use first passage times

34

Same Example as Before

0.25

0.2

There are several possibilities: find the gene that

(011)

0.15

minimizes the mean first passage time maximizes the probability of reaching a particular state before a certain fixed time
(101)

0.1 (110)

0.05

0 0

10 K 0

12

14

16

18

20

minimizes the time needed to reach a certain state with a given fixed probability.
35

Sensitivity of Stationary Distributions to Gene Perturbations What is the effect of perturbations on long-term network behavior? Similar problems have been addressed in perturbation theory of stochastic matrices. Using recent results by Cho & Meyer (2000), we can show
36

Sensitivity Result

One important implication is that if a particular state of a PBN can be easily reached from other states, meaning that the mean first passage times are small, then its steady-state probability will be relatively unaffected by perturbations. Such sets of states, if we hypothesize them to correspond to some functional cellular states, are thus relatively insensitive to random gene perturbations.

37

Current work in progress

Robust inference of PBNs from data. Designing small sub-networks from data. Steady-state analysis using MCMC-type methods.
l

Diagnosing convergence, establishing a priori bounds on convergence using the structure of the network.

Manipulating network structure to alter longterm behavior in a desired way.

38

Subnetwork generated by Dr. Ronaldo Hashimoto

39

Current work in progress

Robust inference of PBNs from data. Designing small sub-networks from data. Steady-state analysis using MCMC-type methods.
l

Diagnosing convergence, establishing a priori bounds on convergence using the structure of the network.

Manipulating network structure to alter longterm behavior in a desired way.

40

41