BATS: Budget-Constrained Autoscaling for Cloud
Performance Optimization
A. Hasan Mahmud Yuxiong He
Florida International University Microsoft Research
1. INTRODUCTION
In recent years, cloud computing has experienced an ex-
ponential growth, providing a wide range of services such as
scientific computing and web hosting. Cloud service users
desire two major benefits: good performance and low opera-
tional costs. Autoscaling is a key property of the cloud sys-
tems that aims to achieve this goal. Using autoscaling, users
can dynamically request more resources (e.g., virtual ma-
chines, or VMs) and pay more when their demand increases,
and vice versa. In this paper, we propose to optimize cloud
application performance by leveraging autoscaling while sat-
isfying a long-term budget constraint (e.g., monthly or yearly
budget). Such budget constraints are commonly applied to
businesses, universities and governments, which typically al-
locate annual IT operational budgets at the beginning of
each fiscal year [6]. We focus on delay-sensitive cloud appli-
cations (e.g., web services) where application performance
is measured by the delay of responses. While optimizing de-
lay performance under a budget constraint is important for
cloud users, it is practically challenging. Requesting more
VMs at the current time will reduce the available budget
for future uses and hence, optimally scaling VM acquisi-
tions requires the complete offline information (e.g., future
workload demand) over the entire budgeting period, which
is very difficult, if not impossible, to obtain in advance, es-
pecially considering highly dynamic workloads and possible
traffic spikes due to breaking events.
The default autoscaling mechanisms offered by major cloud
service providers, such as Amazon EC2 and Windows Azure,
typically scale VM instances based on resource utilization
indicators such as CPU and memory usage: e.g., adding
a new VM instance or switching to a bigger VM instance
that has more virtual CPU cores when the current CPU
utilization exceeds a certain user-specified threshold [1, 3].
However, when considering the budget constraint, it is dif-
ficult to decide the optimal resource usage threshold a pri-
ori, because the threshold value depends on the user bud-
get and the workloads during the entire budgeting period
(e.g., setting a low threshold increases performance but may
Permission to make digital or hard copies of part or all of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for profit or commercial advantage, and that copies bear this notice and the full ci-
tation on the first page. Copyrights for third-party components of this work must be
honored. For all other uses, contact the owner/author(s). Copyright is held by the
author/owner(s).
SIGMETRICS’14, June 16–20, 2014, Austin, Texas, USA.
ACM 978-1-4503-2789-3/14/06.
http://dx.doi.org/10.1145/2591971.2592019.
563
Shaolei Ren
Florida International University
violate budget constraint, and vice versa). Recent efforts
on autoscaling for optimizing the performance under long-
term budget constraints have primarily focused on evenly
distributing budgets across time or predicting the long-term
future workloads, neither of which applies to highly-dynamic
delay-sensitive workloads in practice [4, 5].
In this paper, in view of the practical difficulty in accu-
rately predicting long-term future workloads, we develop an
online autoscaling system, called BATS (Budget-constrained
AuToScaling), that dynamically scales VM instances to op-
timize the delay performance while satisfying user budget
constraint in the long run. The core of BATS is an online
autoscaling algorithm, which only requires the past and in-
stantaneous workload information to make effective scaling
decisions. The key idea of our algorithm is to keep track
of the budget deficit online and incorporate it into the on-
line autoscaling decision: if the actual VM expenses exceed
the expected cost thus far, BATS will request fewer and/or
smaller VM instances subject to the minimum delay per-
formance requirement, such that the budget deficit can be
decreased. Thus, the budget deficit tracked online serves as
a feedback mechanism to guide BATS towards satisfying the
long-term budget constraint.
The contribution of our work is that we provide a full-
fledged autoscaling solution to optimize delay performance
while meeting users’ long-term budget constraints that widely
exist in practice. In particular, we offer a provably-efficient
autoscaling algorithm, a user-friendly automated system im-
plemented on Windows Azure, and comprehensive perfor-
mance evaluation showing advantages of the proposed so-
lution compared with state of the art using both system
implementation and simulation.
2. SYSTEM IMPLEMENTATION
We describe the software architecture of BATS autoscaler.
It has three main modules: (1)Monitor : gathering differ-
ent performance metrics or resource utilization indicators of
the hosted cloud application, such as number of web server
connections, (2) Scheduler : deciding the optimal number
of VMs for the cloud application based on BATS algorithm
with a goal of minimizing delay while meeting the budget
constraint, and (3) Scaler : executing the scheduler deci-
sion and submits the scaling request to Windows Azure.
The Scheduler has two submodules that uses the output
from the Monitor : a Predictor that predicts the workload
arrival using an auto-regressive model and a Performance
Watcher that detects the current workload status. Perfor-
mance Watcher ensures that autoscaler meets the delay per-
formance goal in case of workload prediction error or sudden 1.5
workload spike. While we use Windows Azure as an exam- BATS 10 BATS

Average Delay (sec)

EqualSC EqualSC
ple, the modular design of our autoscaler is also applicable

Total Cost ($)

OptOffline 8 OptOffline
for other cloud platforms such as Amazone EC2 by modify- 1 ReactSC 6 ReactSC
ing the Scaler and Monitor modules.
4
2
3. EXPERIMENTAL RESULT 0.5

First, we discuss our experimental setup. We consider a 0

10 20 30 40 10 20 30 40
time slotted model and generate workload arrivals based on Time Slot Time Slot
the MSR (Microsoft Research) workload trace to drive our (a) Average delay. (b) Total Cost.
experiment. To limit lengthy experiment, the budgeting pe-

Instantaneous Delay (sec)

riod in our study consists of 48 time slots and the duration 2.5 0.8

Instantaneous Cost ($)

of each time slot is 1 hour. The default budget for our exper-
iment is $8.5 while the cost for one VM instance per hour is
$0.02. We deploy the RUBiS [2] benchmarking application
into the web role of Azure Cloud Service and scale the un-
derlying resources by varying the number of VM instances.
We use extra-small VM instances. The performance goal is
to achieve an average delay of 520ms while the maximum
tolerable average delay is 1500ms. To make the optimal
scaling decision (i.e., number of VM instances) at each time
slot, BATS needs to determine the resulting delay for the
incoming workload under various scaling configuration (i.e.,
number of VM instances). We model delay of RUBiS work-
load using a delay lookup table. Specifically, we vary the
number of clients from the workload generator and obtain
average delay for each different scaling configuration. Thus,
we build the delay lookup table for various combinations of
workload arrival rates and numbers of VM instances.
We now describe three benchmarks to compare BATS:
(1) EqualSC: evenly dividing the available budget across
all the time slots and obtains the number of VM instances
that can be reserved for the entire budgeting period based
on discounted pricing (20% for reserved instances [3]), (2)
ReactSC: widely adopted autoscaler [1, 3] that constantly
monitors the average workload arrival rates measured over
the last 5 minutes and uses the delay lookup table to deter-
mine the minimum number of VM instances such that the
resulting delay is less than a specified threshold (520ms),
and (3) OptOffline: optimally dividing the whole budget
among time slots with perfect workload arrival information
over the entire budgeting period.
Next, we compare the performance of BATS with bench-
mark algorithms and show cumulative average delay and cu-
mulative cost in Fig. 1(a) and Fig. 1(b), respectively. The
cumulative average value for a time slot t is the correspond-
ing average value of time slot 0 to t. We show the average
delay performance and cost comparisons for each individ-
ual time slot in Fig. 1(c) and Fig. 1(d), respectively. As
shown in Fig. 1(a), BATS reduces delay by 34% compared to
EqualSC while achieving the same budget constraint, even
though EqualSC receives discounted pricing. This is mainly
because EqualSC evenly divides the budget across each time
slot and reserves 11 VM instances without considering the
workload variation. As a result, when there is a workload
spike (e.g., in the 4th time slot), the delay becomes very
large, as shown in Fig. 1(c). The average delay reduction
of BATS is 10% compared to ReactSC. The degrading per-
formance of ReactSC comes from the long lagging time: it
takes up to 5 minutes to detect the system status change
(e.g., workload variation) and even after detection, it takes
up to 10 minutes to acquire a new VM instance. During the
lagging time, all the incoming workloads experience longer
BATS BATS
EqualSC EqualSC
2 0.6
OptOffline OptOffline
1.5 ReactSC ReactSC
0.4
1
0.2
0.5
0
10 20 30 40 10 20 30 40
Time Slot Time Slot
(c) Instantaneous delay. (d) Instantaneous Cost.
Figure 1: Comparing BATS with other algorithms.
average delays. This demonstrates the importance of pro-
actively predicting the near-future (e.g., hour-ahead) work-
loads as used by BATS, thereby highlighting the limitations
of reactive autoscaling rules. Moreover, Fig. 1(b) shows that
the cost saving of BATS is 10% compared to ReactSC. It is
mainly because ReactSC ignores the budget constraint and
always makes scaling decisions such that resulting average
delay equals 520ms. ReactSC also violates $8.5 budget con-
straint. We also observe that the average delay of BATS
is very close to OptOffline (with a difference less than 4%),
while Fig. 1(b) shows that the cost is almost the same. Thus,
the results demonstrate the effectiveness of BATS.
Finally, we summarize the simulation study that demon-
strates the benefits and robustness of BATS on a wide range
of scenarios. Even with high workload prediction errors,
the performance of BATS is still robust: e.g., for 20% pre-
diction error, the resulting average delay only increases by
1.4%. Moreover, by leveraging both horizontal and vertical
scaling, BATS can further improve delay performance while
satisfying the budget constraint. BATS can also incorpo-
rate reserved instances in its operation to lower operational
cost and improve delay performance. The simulation study
for other workload traces also demonstrates that BATS per-
forms better even under fairly smooth workloads.
4. REFERENCES
[1] Amazon ec2. http://aws.amazon.com/autoscaling/.
[2] Rubis. http://rubis.ow2.org/.
[3] Windows azure. http://www.windowsazure.com/.
[4] M. Malawski, G. Juve, E. Deelman, and J. Nabrzyski.
Cost-and deadline-constrained provisioning for scientific
workflow ensembles in iaas clouds. In SC, 2012.
[5] M. Mao, J. Li, and M. Humphrey. Cloud auto-scaling with
deadline and budget constraints. In GRID, 2010.
[6] S. VanRoekel. The FY14 President’s IT budget: Innovate,
deliver, protect, https://cio.gov/
the-fy14-presidents-it-budget-innovate-deliver-protect/.

564