BROUGHT TO YOU IN PARTNERSHIP WITH

CONTENTS

öö Introduction to Serverless
Computing

Introduction to ö ö Common Use Cases for Serverless

ö ö What Is Monitoring? What Is

Observability?

Serverless Monitoring ö ö Detecting Performance Problems

in Serverless

ö ö Implementation of Monitoring

ö ö Glossary

WRITTEN BY EMRAH SAMDAN

VP OF PRODUCTS AT THUNDRA

Introduction to Serverless Computing However, in exchange for the pay-as-you-go service and simplicity
Cloud migration has enabled software companies to outsource in operations, software developers lost the ability to monitor their
their old-school, in-house servers to cloud providers thanks to an serverless applications. This is due to developers uploading their code
infrastructure-as-a-service approach. After that, IT admins and onto cloud platforms that do not provide access and control of the cloud-
operations people continued to manage the provisioning, patching, based environment. This Refcard looks into challenges of monitoring
and securing of their cloud servers. and observability within serverless architectures and explains how a
monitoring system for a serverless architecture can be implemented.
Eventually, cloud providers came up with a new cloud execution model
where they fully manage the provisioning and the allocation of these
servers. In this model, called function-as-a-service or serverless, cloud
Common Use Cases for Serverless
As the concept of serverless propagates through the software
vendors run your applications in a stateless, ephemeral container
industry, applications of it are seemingly boundless. Serverless now
when triggered by an event.
enjoys center stage in various fields that range from APIs all the way
There are many different event triggers, such as a direct API call, a to machine learning.
message written to a queue, a signal sent by an IoT device, and so on.
FaaS arose as a business model thanks to its pay-as-you-go billing
method. With this, cloud providers charge their customers only for the
time the function is running. This is particularly useful for companies
that have unpredictable traffic because they can scale automatically,
and they don't pay for idle resources.

AWS announced its Lambda service as a FaaS solution back in

November 2014. Azure and Google Cloud Functions followed in early
2016 and March 2017 respectively.

1

This is because serverless is not only mitigating the pains associated
with managing personal servers but is also providing cost-effective
and scalable solutions to the industry. Described below are some of
the use cases with serverless (this is not intended as an exhaustive list).
API CREATION AND MANAGEMENT
One of the most popular use cases regarding serverless technology is
to build API services. A fundamental problem with APIs is scalability,
hence developers are choosing serverless due to its scalability and
pay-as-you-go capabilities.
Today, there are over 21,200 APIs in the world, all with different traffic
needs. By having an API on a serverless architecture, developers can
ensure that if there are sudden spikes in traffic, the resources will scale
accordingly to meet the needs of the incoming requests. Furthermore,
providers of these APIs can greatly cut costs as serverless adopts the
pay-as-you-go ideology — if an API has no traffic, no resources will be
consumed by the API, so there are no charges. These advantages make
serverless a favored tool among new startups, as reported in Digital
Ocean's quarterly report.
IOT SUPPORT
Internet of Things is yet another sector of technology exploring
serverless. This is because IoT is aimed at "dumb" devices connecting
to web-based resources. The simplicity and scalability provided by
serverless is appealing.
Unlike conventional APIs, IoT devices do not see many requests being
made to their web endpoints. For this reason, having dedicated
resources leads to unnecessary costs. Since there are no charges when
there are no requests, this makes serverless an ideal choice.
Furthermore, setting up a serverless architecture allows developers
and product managers to abstract the server-side layer away. That
means IoT providers can simply focus on producing devices and busi-
ness logic on the web. Using IoT with serverless is quite rudimentary,
INTRODUCTION TO SERVERLESS MONITORING
training can be broken down and fit into different serverless functions,
with each one scaling accordingly to meet the computational needs of
that specific step. Therefore, engineers can ensure that resources are
not wasted and costs are managed effectively.
Many argue that machine learning engineers can achieve the same
models by using containers without worrying about the limitations
that the young serverless technologies currently impose. However, the
complexity involved in setting up and managing these containers can be
even more daunting than developing the machine learning model itself.
Serverless alleviates these complexities, allowing machine learning
engineers to focus solely on their models. The other alternative is pro-
visioning servers, which only adds additional costs and the complexity
of initializing and managing the infrastructure. Considering the options
available, serverless has provided a simple alternative to machine learn-
ing engineers, allowing them to focus more on breaking the bounds of
computational science rather than fixing the links between their servers.
What Is Monitoring? What Is Observability?
Monitoring has always been in the life of operations people as the act
of keeping an eye on the specific set of metrics to ensure the health of
the system. Observability, on the other hand, is the state of the system
in which ops people can go deeper into their system when facing an
issue in production.
To summarize, people can monitor their systems while a system can
be observable. In this sense, observability is regarded as the superset
of monitoring. In her book Distributed Observability, Cindy Sridharan
summarizes the relation.



 
as described by Andy Warzon, the founder and CTO of Trek10. 
 



 




Additionally, cloud providers, such as AWS and Azure, provide reliable
and trusted services to build IoT applications. Due to the reliability
and security provided within these provided environments as well as 
the continuous improvements attributed to the competition in the
field, developing and providing IoT applications becomes easier, more
Observability requires instrumentation that emits actionable informa-
cost-effective, and more reliable.
tion out of system behavior. By instrumenting the code, developers

MACHINE LEARNING
Machine learning engineers face a daunting task even before they
write their first line of code: setting up their environment to run their
code. However, with serverless, the complexity of the environment
setup can now be offloaded to cloud providers.
Additionally, the architecture that serverless implies also benefits ma-
chine learning models when being developed. Various steps in model
aim to understand the problems with traces, metrics, and logs when
something unexpected happens.
Developers can instrument their code manually by printing logs and
injecting spans to the necessary places, or automatically by using
third-party tools or even by writing their own tools to gather a specific
set of information by injecting monitoring code either at runtime or at
compile time.

3 BROUGHT TO YOU IN PARTNERSHIP WITH

 INTRODUCTION TO SERVERLESS MONITORING

THREE PILLARS OF OBSERVABILITY
In order to make a system observable, developers need the three
pillars of observability to promote the transparency required to under-
stand how a system is working. They are not interchangeable and are
effective when all of them are present for developers and ops people.
another transaction starting in the first Lambda. For this reason, it is
very hard to detect a problem that occurred in this chain of invoca-
tions. There are some vendor-specific solutions for distributed tracing,
but it mostly requires developers to write instrumentation code.

1. Metrics give you an initial idea of the overall health of your

system. The metrics that are important can vary depending on
the system. For example, invocation duration is an important
metric because a jump in invocation duration usually means an
unhealthy condition in the serverless architecture. Metrics are
the starting point of the investigation because they signal devel-
opers when an unexpected situation occurs. However, a detailed
analysis requires other pillars.

2. Traces enable developers to measure the time spent in various

places of their applications. Using traces, developers have an
overview of how their system is behaving. Local tracing shows
the measurement of time in a closed system, i.e. single server-
less functions, while distributed tracing shows the measurement
of time in a distributed architecture. This is particularly useful in
serverless environments.

3. Logs constitute the third and oldest of the observability pillars.

It is basically the reflection of print statements while you're
debugging. It can be helpful while solving an issue, but it is
advised to have different log levels and structured logging in
order to find what you're looking for in a pile of logs. Even then,
developers need to implement sampling methods in order to
reduce the logging throughput of their systems for faster debug-
ging and minimizing storage size.
All three pillars of observability are required for meaningful and action-
able insights. However, monitoring serverless applications is difficult,
as you cannot add monitoring tools into the cloud environment due
to the black-box nature of serverless. As a result, your monitoring tool
needs to run alongside your serverless functions.

THE ROLE OF OBSERVABILITY AND MONITORING IN SERVERLESS
Monitoring in a serverless architecture can be cumbersome with met-
rics limited to what the cloud provider makes available.
For example, AWS Lambda only provides invocation counts, durations,
error ratios, concurrent executions, and throttles. There are also logs
that you can check — but discovering an issue with logs is like looking
for a needle in a haystack unless you have structured logging.
Serverless transactions are different from microservices because
they are event-driven by nature. Transactions are mainly a chain of
asynchronous invocations of serverless functions. In such a case,
distributed tracing is crucial to understanding the full lifecycle of a
serverless transaction.
The following image depicts a serverless transaction consisting of
seven AWS Lambda functions, an SNS topic, an SQS queue, a Kinesis
Stream, an API Gateway route, an S3 bucket, and a Firehose stream.
When a transaction ends in the last Lambda function, there might be
This stresses the need for a lightweight monitoring tool that adds little
overhead to your primary functions before publishing the collected
monitoring data. This data must be sent and processed by a backend
service, but this can only be done at the end of the main function exe-
cution. This means that the serverless platform should not close after
the serverless function has been executed, but rather after the moni-
toring data has been sent. This can be hazardous to customer-facing
applications and result in longer invocation durations.
SERVERLESS MONITORING AND OBSERVABILITY USE CASES
Although the computing model shifts to a completely new paradigm
with serverless, the monitoring and observability objectives haven't
changed. Either with monitoring reactively or by observability proac-
tively, the first objective is to avoid customer dissatisfaction due to
performance degradation or an outage.
Some sub-objectives can be listed as understanding the root cause
of errors, detecting bottlenecks, reducing MTTR, and so on. Moreover,

4 BROUGHT TO YOU IN PARTNERSHIP WITH


another objective arose from the serverless pay-as-you-go model:
controlling the cost. Serverless systems can be more robust because
they're based on managed serverless components, but they have the
potential to be expensive when not used properly.
For this reason, monitoring use cases can be grouped into the
category of either detecting performance problems in the times of
debugging, troubleshooting, and monitoring or controlling costs for
serverless transactions.
Detecting Performance Problems in Serverless
Serverless architectures have performance problems unique to
themselves. These include cold starts, timeouts, and long-running
functions due to performance problems associated with downstream
services. Traditional problems due to bad coding practices still exist in
serverless architectures.
Cold starts occur when a function is invoked for the first time. It takes
time to initialize the container and load the dependencies that your
function requires. The frequency and duration of cold starts depends
on the programming language you choose, the memory configuration
of your Lambda function, and its security configuration. Cold starts
are inevitable, but their frequency and duration can be reduced by
warm-up modules provided by the community. For customer-facing
latency issues, it's important to determine if the culprit is a cold start
or another problem.
Timeouts are also critical in the performance of serverless applications.
AWS Lambdas can execute up to 15 minutes, but it is advised that users
set the timeout limit to 10 seconds. This is due to the general belief
that if a service does not respond in the first 10 seconds, it should
be expected to timeout. Cloud vendors alert users if timeouts occur,
and during such an instance, the best practice is to check if there is a
problem with downstream services and apply some circuit breakers
limiting interaction with the problematic service.
Timeouts can usually be expected due to CPU intensive operations,
but the CPU-intensive part of your function can be difficult to detect.
Therefore, local tracing and/or CPU/memory profiling are required to
analyze what parts of the functions are using system resources and
contributing to the invocation duration.
Although the serverless computing model is useful because of pay-as-
you-go billing, it's important to keep your eye on the costs attached
with serverless architectures. The cost of serverless functions depends
on the total duration and allocated memory for the serverless function.
Hence, developers should work on decreasing the invocation duration
or decreasing the memory when possible to control cost att ached with
serverless. Note that invocation duration depends on the allocated
memory, as well, because more memory means more CPU power
for Lambda functions, which eventually may mean less invocation
5 BROUGHT TO YOU IN PARTNERSHIP WITH
INTRODUCTION TO SERVERLESS MONITORING
duration. Therefore, developers should fine-tune their allocated
memory by monitoring the memory and CPU usage of their functions.
Moreover, they aim to decrease the time spent with the function by
managing external services carefully and warming up containers to
avoid cold starts. For this purpose, they need to closely monitor their
serverless architecture and external services.
Implementation of Monitoring
Although cloud vendors provide their own monitoring systems,
developers usually need third-party implementations in order to have
actionable details to decrease the mean time to resolve (MTTR) in a
serverless architecture. Instrumentation, either manual or automated,
is critical to having all three pillars of observability achieve an
aggregated view of your system.
Instrumentation is generally achieved by adding a library that
automatically injects code or lets the developer inject instrumentation
because it is not possible to put an agent where the function runs (as
it runs in a different container at every invocation). Adding this library
as a Lambda Layer is the AWS-specific way of doing so, which eases the
monitoring. However, these libraries generate large amounts of data
that eventually becomes hard to store and process. For this reason,
sampling plays an important role to pursue a healthy monitoring system.
WHAT TO WATCH FOR THROUGHOUT THE SOFTWARE
DEVELOPMENT CYCLE
Monitoring requirements for serverless applications vary depending on
the application's stage. Developers want to fine-tune their applications
during development and testing before they proceed to production.
Traces and logs are more important at this stage in order to see the
potential bottlenecks before they become serious in production.
Running an instrumented serverless architecture in a development en-
vironment on a cloud vendor reveals the problematic issues. After going
serverless, traces and logs are still useful for troubleshooting erroneous
conditions, while metrics are crucial to staying informed about the gen-
eral system health. A typical example of this is understanding the root
cause by checking the traces and logs causing a jump in the invocation
duration as you get alerted when the duration of a single function and/
or whole serverless transaction exceeds a threshold.
MANUAL AND AUTOMATED INSTRUMENTATION IN SERVERLESS
OBSERVABILITY
Instrumentation is the process whereby serverless architectures are
configured to enable monitoring. That means you would specify what
parts of your architecture you would like to monitor, and depending
on the monitoring tool of your choice, you could perform manual or
automated instrumentation or even a combination of both.
What does this mean? Manual instrumentation is when you yourself
specify the individual parts of your serverless application you would
like monitored. Automatic instrumentation is when you specify the

general pattern of your serverless functions and the tool then monitors
all components that lie within the defined pattern.
Both methods have their respective pros and cons. The method you
choose depends on your monitoring goals. For example, manual
instrumentation allows the ultimate customization of how you would
like to proceed with monitoring your serverless application, but it also
increases the relative setup complexity. On the contrary, automatic
instrumentation greatly reduces the entire instrumentation process,
even leading to a single line of configuration to attain detailed trace
data illustrating the behavior of your serverless functions.
However, you may not achieve the detailed and customized monitor-
ing you are looking for, so the solution is usually a combination of both
methods to achieve overall observability of your serverless archi-
tecture, with manual instrumentation providing the more in-depth
analysis of principal functions.
HOW TO ADD MONITORING AS LAMBDA LAYERS
AWS introduced Lambda Layers at the end of 2018 at the AWS re:Invent
conference. Since then, tasks such as monitoring have become simpler,
as serverless developers can now add the monitoring tool of their
choice as a Lambda Layer. Essentially, a Layer is a zipped archive that
can contain anything from required libraries to custom runtimes. This
package, in the form of a Lambda Layer, can then be shared across
multiple functions once it is loaded into your AWS Lambda console.
With Lambda Layers, configuring your monitoring tools has become
easier, as you now simply import your monitoring tool in the form of a
Layer on top of your Lambda function. You can further configure your
monitoring tool with environment variables with no need for manual
configuration in the code itself.
In AWS, monitoring tools can be added to your Lambda environment
via the Lambda console. Within the console, you can click on Layers in
the Designer view and simply add the packaged monitoring library of
your choice. After ensuring that the correct layer version and package
have been loaded, you can initialize your monitoring tool with environ-
ment variables. That means, depending on the monitoring tool you are
using, you can pass in your monitoring credentials, perform instru-
mentation, and even enable sampling of your data. You can achieve
comprehensive observability without adding a single line of code.
6 BROUGHT TO YOU IN PARTNERSHIP WITH
INTRODUCTION TO SERVERLESS MONITORING
SYNCHRONOUS VS. ASYNCHRONOUS MONITORING
There are two ways you can publish your monitoring data: synchronously
or asynchronously. There are various reasons why you may choose to
publish monitoring data synchronously or asynchronously. However,
according to a whitepaper by AWS released in November 2017, it is
recommended that monitoring data be published asynchronously.
This is because synchronous publication monitoring data can lead
to overhead in your serverless functions; it entails longer execution
times since your monitoring data has to be returned at the end of
your invocation. Other reasons that could be problematic are if the
publishing failed or if you are behind a secure environment such as
an Amazon VPC. That would mean that your monitoring tool cannot
access the generated data, and overall, you have either unreliable or
potentially no monitoring data available.
In that case, it is advised to monitor your serverless architecture
asynchronously by taking advantage of the cloud vendor's data
logging capabilities. For example, your monitoring data could be
transferred to AWS CloudWatch, and regardless of the execution
duration or structure of your serverless function, your monitoring
tool can always access the monitoring data from CloudWatch. Even
though synchronous monitoring is simple and easy to configure,
asynchronous monitoring has major benefits that should not
be ignored. The following image shows the implementation of
asynchronous monitoring with AWS Lambda.
SAMPLING MONITORING DATA
Monitoring serverless applications can be costly, especially when
trying to achieve the three pillars of observability. Moreover, using
structured logs to make searching easier further exasperates costs
associated with logging. To tackle this problem, it is advised to
implement a sampling mechanism that reduces the amount of
monitoring data reported. Sampling is wise because not every single
trace, metric, and log is necessary for every single invocation of a
serverless architecture for keeping the system healthy.
The question here is how to sample the monitoring data. Traditional
approaches offer rule-based systems that only record, for example,
one out of every 1,000 invocations or one invocation every 10 minutes.

However, you may miss important invocations with rule-based
sampling. In order to reduce stored data for serverless monitoring
effectively, you must apply intelligent sampling rules.
For example, you only need the traces when the duration exceeds a
certain threshold or a specific type of error occurs. An intelligent sam-
pling system should be able to:
• Filter out non-erroneous invocations and gather every detail
about problematic invocations with specific error types.
• Filter out invocations with expected durations and gather every
detail about long-running invocations to reveal bottlenecks.
Glossary
• AWS X-ray: Application management tool provided by
AWS to allow you to monitor your cloud systems, including
serverless applications.
• Cold start: The process whereby a serverless container is
initialized upon invocation of a function. This setup of these
environments leads to latency between the invocation being
sent and the actual execution of the function.
• Function: A block of executable code that is uploaded
to a functions-as-a-service platform. These functions are
usually event-driven as they are executed on the occurrence
of another event or run periodically depending on your
configuration.
Written by Emrah Samdan, VP of Products at Thundra
Emrah Samdan is VP of Products at Thundra. Organization committee for Serverless Turkey and ServerlessDays
Istanbul. Helping serverless developers have observability on their applications.
DZone communities deliver over 6 million pages each month
to more than 3.3 million software developers, architects,
and decision makers. DZone offers something for everyone,
including news, tutorials, cheat sheets, research guides, fea-
ture articles, source code, and more. "DZone is a developer’s
dream," says PC Magazine.
7 BROUGHT TO YOU IN PARTNERSHIP WITH
INTRODUCTION TO SERVERLESS MONITORING
• Invocation: An execution of a serverless function; either
triggered by other events or periodically.
• Lambda Layers: A bundled package of libraries, dependencies,
and/or runtimes that can be incorporated into your Lambda
functions separately and shared across multiple functions.
Lambda Layers are provided by AWS and were introduced at
AWS re:Invent 2018.
• Log: The print statements and messages logging agents used
in a codebase. These logs could be messages pertaining to
relevant errors, simple print statements, or serialized output.
• Metrics: Statistical data related to the computation and
performance of a function. These statistics could range from
information about cold starts and erroneous invocations, all
the way to average memory usage and CPU usage throughout
your serverless architecture.
• Trace: A representation charting out single functional events
in the form of a trace span. Events that are a result of other
events are represented as child spans below the main event.
For example, a function calls another function, and this is
represented as the calling function being the parent span and
the called function being the child span, and their behavior
and data transactions are mapped out accordingly.
• Trace chart: A graphical representation of trace data that
shows how much time each part of the system takes to
complete during function execution.
Devada, Inc.
600 Park Offices Drive
Suite 150
Research Triangle Park, NC
888.678.0399 919.678.0300
Copyright © 2019 DZone, Inc. All rights reserved. No part of this
publication may be reproduced, stored in a retrieval system, or
transmitted, in any form or by means electronic, mechanical,
photocopying, or otherwise, without prior written permission of
the publisher.