Scribd Logo
Fazer o upload

Bem-vindo(a) ao Scribd!

  • An Approach For Profiling Distributed Applications Through Network Traffic Analysis (Apresentação)

    Enviado por

    Vinicius Cardoso Garcia
    175 visualizações53 páginas
    Trabalho apresentado ao Programa de Pós-graduação em Ciência da Computação do Centro de Informática da Universidade Federal de Pernambuco como requisito parcial para obtenção do grau de Mestre em Ciência da Computação. Resumo: Distributed systems has been adopted for building modern Internet services and cloud computing infrastructure, in order to obtain services with high performance, scalability and reliability. Cloud computing SLAs require low time to identify, diagnose and solve problems in a cloud computing production infrastructure, in order to avoid problem impacts into the quality of service provided for its clients. Thus, the detection of error causes, diagnose and reproduction of errors are challenges that motivate efforts to the development of less intrusive mechanisms for monitoring and debugging distributed applications at runtime. Network traffic analysis is one option to the distributed systems measurement, al- though there are limitations on capacity to process large amounts of network traffic in short time, and on scalability to process network traffic where there is variation of resource demand. The goal of this dissertation is to analyse the processing capacity problem for measur- ing distributed systems through network traffic analysis, in order to evaluate distributed systems at a data center, using commodity hardware and cloud computing services, in a minimally intrusive way. We proposed a new approach based on MapReduce, for deep inspection of dis- tributed application traffic, in order to evaluate the behavior of distributed systems at runtime, using commodity hardware. In this dissertation we evaluated the effectiveness of MapReduce for a deep packet inspection algorithm, its processing capacity, completion time speedup, processing capacity scalability, and the behavior followed by MapReduce phases, when applied to deep packet inspection for extracting indicators of distributed applications.

    Título original

    An Approach for Profiling Distributed Applications Through Network Traffic Analysis (Apresentação)

    Direitos autorais

    © Attribution Non-Commercial ShareAlike (BY-NC-SA)

    Formatos disponíveis

    PPT, PDF, TXT ou leia online no Scribd

    Você considera este documento útil?

    Este conteúdo é inapropriado?

    Denunciar este documento
    Trabalho apresentado ao Programa de Pós-graduação em Ciência da Computação do Centro de Informática da Universidade Federal de Pernambuco como requisito parcial para obtenção do grau de Mestre em Ciência da Computação. Resumo: Distributed systems has been adopted for building modern Internet services and cloud computing infrastructure, in order to obtain services with high performance, scalability and reliability. Cloud computing SLAs require low time to identify, diagnose and solve problems in a cloud computing production infrastructure, in order to avoid problem impacts into the quality of service provided for its clients. Thus, the detection of error causes, diagnose and reproduction of errors are challenges that motivate efforts to the development of less intrusive mechanisms for monitoring and debugging distributed applications at runtime. Network traffic analysis is one option to the distributed systems measurement, al- though there are limitations on capacity to process large amounts of network traffic in short time, and on scalability to process network traffic where there is variation of resource demand. The goal of this dissertation is to analyse the processing capacity problem for measur- ing distributed systems through network traffic analysis, in order to evaluate distributed systems at a data center, using commodity hardware and cloud computing services, in a minimally intrusive way. We proposed a new approach based on MapReduce, for deep inspection of dis- tributed application traffic, in order to evaluate the behavior of distributed systems at runtime, using commodity hardware. In this dissertation we evaluated the effectiveness of MapReduce for a deep packet inspection algorithm, its processing capacity, completion time speedup, processing capacity scalability, and the behavior followed by MapReduce phases, when applied to deep packet inspection for extracting indicators of distributed applications.

    Direitos autorais:

    Attribution Non-Commercial ShareAlike (BY-NC-SA)
    Baixe no formato PPT, PDF, TXT ou leia online no Scribd
    Sinalizar o conteúdo como inadequado
    175 visualizações53 páginas

    An Approach For Profiling Distributed Applications Through Network Traffic Analysis (Apresentação)

    Enviado por

    Vinicius Cardoso Garcia
    Trabalho apresentado ao Programa de Pós-graduação em Ciência da Computação do Centro de Informática da Universidade Federal de Pernambuco como requisito parcial para obtenção do grau de Mestre em Ciência da Computação. Resumo: Distributed systems has been adopted for building modern Internet services and cloud computing infrastructure, in order to obtain services with high performance, scalability and reliability. Cloud computing SLAs require low time to identify, diagnose and solve problems in a cloud computing production infrastructure, in order to avoid problem impacts into the quality of service provided for its clients. Thus, the detection of error causes, diagnose and reproduction of errors are challenges that motivate efforts to the development of less intrusive mechanisms for monitoring and debugging distributed applications at runtime. Network traffic analysis is one option to the distributed systems measurement, al- though there are limitations on capacity to process large amounts of network traffic in short time, and on scalability to process network traffic where there is variation of resource demand. The goal of this dissertation is to analyse the processing capacity problem for measur- ing distributed systems through network traffic analysis, in order to evaluate distributed systems at a data center, using commodity hardware and cloud computing services, in a minimally intrusive way. We proposed a new approach based on MapReduce, for deep inspection of dis- tributed application traffic, in order to evaluate the behavior of distributed systems at runtime, using commodity hardware. In this dissertation we evaluated the effectiveness of MapReduce for a deep packet inspection algorithm, its processing capacity, completion time speedup, processing capacity scalability, and the behavior followed by MapReduce phases, when applied to deep packet inspection for extracting indicators of distributed applications.

    Direitos autorais:

    Attribution Non-Commercial ShareAlike (BY-NC-SA)
    Baixe no formato PPT, PDF, TXT ou leia online no Scribd
    Sinalizar o conteúdo como inadequado
    de 53

    An Approach for Profiling Distributed

    Applications Through Network Traffic


    Analysis
    Thiago Vieira
    tpbv@cin.ufpe.br
    http://github.com/tpbvieira
    CIn UFPE

    Outline

    Motivation and Problem Statement


    Background and Related Work
    Profiling Distributed Applications Through
    Deep Packet Inspection
    Evaluating MapReduce for Network
    Traffic Analysis
    Conclusion

    Motivation
    and
    Problem Statement
    3

    Motivation

    Motivation

    low intrusion, high accuracy and fast results??


    How??
    5

    Motivation

    Protocols, flows, throughput and load distribution

    Motivation

    Online Classification
    App. State Evaluation = More processing
    7

    Motivation

    MapReduce restrictions
    MapReduce for DPI?
    8

    Problem Statement

    How to evaluate large application


    traffic of a datacenter?
    Can MapReduce express DPI
    algorithms?
    How is the MapReduce processing
    capacity for profiling application traffic?

    Problem Statement

    Our Goal:

    To propose a solution for profiling


    applications traffic at a data center

    To analyse the MapReduce


    processing capacity for profiling
    applications through network traffic
    analysis

    10

    Background
    and
    Related Work
    11

    Background

    MapReduce
    Block

    Block

    Block

    12

    Background

    Hadoop specific input data

    New input type and extensions

    Splitting

    Specific configurations

    13

    Related Work

    Lee et al. (2011)

    Hadoop-based packet trace


    processing tool to process network
    traffic.

    Packet level evaluation

    Input Blocks Packets Map

    Is not able to evaluate more than one


    packet per MapReduce iteration

    Inefficient for DPI


    14

    Profiling Distributed
    Applications Through
    Deep Packet Inspection

    15

    Proposal

    A solution for profiling application


    traffic
    Monitoring plans
    Cumulative traces for offline processing
    MapReduce

    Processing capacity and scalability

    Splitting approach

    16

    Architecture

    17

    Architecture

    18

    Evaluation Methodology

    Methodology GQM (Basili et al.,


    1994) and the systematic approach to
    performance evaluation (Jain, 1991)
    Goal To evaluate the effectiveness
    and scalability of MapReduce for
    profilinng application traffic
    AppAnalyzer and AppParser

    19

    Evaluation Methodology

    Q1 Can MapReduce express DPI


    algorithms and extracts application
    indicators from network traffic of
    distributed applications?
    Q2 Is the completion time scalability
    of MapReduce for DPI proportional to
    the number of worker nodes?

    20

    Evaluation Methodology

    21

    Evaluation Methodology

    22

    Experiment Setup

    Traffic from a JXTA-based backup


    system
    Virtual Machines of Amazon EC2
    Indicators:

    Round-trip time

    Connection requisitions per time

    Messages received per time


    Measured the job completion time
    Hadoop default configuration
    23

    Results

    rejects the null hypothesis H0num.indct

    24

    Results

    25

    Results

    rejected the null hypothesis H0scale.prop and confirmed the H1scale.prop

    26

    Possible Threats to Validity

    Small cluster and input size

    According to real traces, the majority


    of the jobs are small and executed
    into small number of nodes

    Fair Scheduler

    27

    Discussion

    MapReduce can express DPI algorithms


    The completion time scalability is not
    proportional to node addition
    Node addition provides different gains
    according to the cluster and input size
    It is necessary more evaluations

    To understand the MapReduce


    behaviour

    How can MapReduce be configured


    for network traffic analysis
    28

    Evaluating MapReduce
    for
    Network Traffic Analysis

    29

    Motivation

    The kind of workload and input type


    submitted for processing by MapReduce
    impacts on its performance

    Lack of evaluation of MapReduce for


    netwokr traffic analysis

    Specific configuration for better


    performance

    30

    Evaluation Methodology

    Methodology Systematic approach to


    performance evaluation (Jain, 1991)
    Goal To investigate the behaviour of
    MapReduce phases, its scalability and
    the speed-up achieved for packet level
    analysis and DPI

    31

    Evaluation Methodology

    32

    Experiment Setup

    Previous network traces, application and


    indicators
    Network traces for packet level
    Physical Machines
    Drivers

    Distributed vs non-distributed

    P3, CountUp and JXTA


    Indicators extracted from Hadoop logs
    by Hadoop-Analyzer
    33

    Results

    34

    Results

    35

    Results

    36

    Results

    37

    Results

    38

    Results

    39

    Results

    40

    Results

    41

    Results

    42

    Results

    43

    Results

    44

    Discussion

    Block size impacts into completion time

    Bigger blocks for smaller clusters

    68MB better than 128MB


    Processing capacity per input size

    Better for bigger data

    More efficient accumulate data

    Input Size + Block Size Pools

    45

    Discussion

    Scalability

    No relevant gains in some cases

    Execution waves
    P3 performs better than CountUpDriver

    Local concurrency
    Phase predominancy

    Map and Shuffle

    Optimizations: Shuffle Start

    46

    Possible Threats to Validity

    P3 version

    2012 February

    Binary version without complete code

    47

    Conclusion

    48

    Conclusion

    MapReduce can express DPI algorithms


    MapReduce scalability is not
    proportional to node addition
    It is necessary to avoid wasting
    resources
    Input, block and cluster size impact into
    completion time
    Map intensive jobs
    It is possible to configure and choose
    the best resource allocation
    49

    Contributions

    An approach to implements DPI


    algorithms through MapReduce
    The parser JNetPCAP-JXTA
    The Hadoop-Analyzer
    Characterization of MapReduce phases
    for packet level analysis and DPI
    Was described the processing capacity
    and scalability of MapReduce for packet
    level analysis and DPI
    50

    Contributions

    Showed the speed-up obtained with


    MapReduce for DPI
    Papers:

    Vieira, T., Soares, P., Machado, M., Assad, R., and Garcia, V.
    Measuring Distributed Applications Through MapReduce and
    Traffic Analysis. In IEEE 18th International Conference
    onParallel and Distributed Systems (ICPADS), 2012
    Vieira, T., Soares, P., Machado, M., Assad, R., and Garcia, V.
    Evaluating Performance of Distributed Systems with
    MapReduce and Network Traffic Analysis. In The Seventh
    International Conference on Software Engineering Advances
    (ICSEA) 2012

    51

    Future Work

    To evaluate all components of the


    proposed architecture
    A technique for efficient evaluation of
    applications through network traffic
    analysis

    52

    Thank you!

    53

    Você também pode gostar