Você está na página 1de 22

Computer Networks 112 (2017) 122143

Contents lists available at ScienceDirect

Computer Networks
journal homepage: www.elsevier.com/locate/comnet

TCP Wave: A new reliable transport approach for future internet


Ahmed Abdelsalam, Michele Luglio, Cesare Roseti, Francesco Zampognaro
Department of Electronics Engineering, University of Rome Tor Vergata, Rome

a r t i c l e

i n f o

Article history:
Received 7 July 2016
Revised 6 October 2016
Accepted 5 November 2016
Available online 8 November 2016
Keywords:
TCP Wave
Future internet
TCP performance

a b s t r a c t
Current and future communication scenarios are very different from those in which TCP was conceived
and developed, bringing new protocol requirements. Performance optimization is usually pursued with
patching of the traditional TCP implementations. Following an alternative approach, TCP Wave has been
designed with the primary goal to satisfy new requirements coming from current networks, such as adaptation to bandwidth/delay changes (due to mobility, dynamic switching, handover), ecient management
of spurious losses as well as link interruption, optimal exploitation of very high link capacity and ecient
transmission of small objects, as in most of Web applications (Web browsing, sensor apps, SCADA, M2M,
etc.), irrespective of the underlying link characteristics. Protocol fairness, TCP friendliness, congestion and
ow control and error recovery are also guaranteed. TCP Wave replaces the window-based transmission
paradigm of the standard TCP with a burst-based transmission, the ACK-clock scheduling with a selfmanaged timer and the RTT-based congestion control loop with an Ack-based Capacity and Congestion
Estimation (ACCE) module. As a valuable study case, the novel TCP Wave capabilities has been validated
on a generic NS-3 simulation testbed where all the most challenging events impacting on transport protocol can occur.
2016 Elsevier B.V. All rights reserved.

1. Introduction
TCP [1] is the traditional protocol conceived to provide a reliable transport service for most of Internet applications, operating
with no knowledge of either underlying network or trac characteristics. TCP was designed with a primary goal: to allow an efcient and reliable management of long connections (mostly long
le transfers) over terrestrial links characterized by small delays,
limited bandwidth and quasi error-free channels [2]. The communication environment had the above characteristics when Internet
was born. TCP eciently satised such requirements, adopting the
TCP window-based transmission, which is still the standard for Internet communications although enhanced and upgraded several
times. Whenever new challenging scenarios emerged (i.e., errorprone wireless links, broadband networks, large delay link), one
or more TCP algorithms (i.e. error recovery, congestion control,
Slow Start, Congestion Avoidance, etc.) have been modied, keeping the overall protocol architecture unchanged (i.e.[39]). The
main TCP transmission principles preserved along several decades
are the window-based transmission paradigm, the ACK-clocked
transmission, the dependency between congestion control and er-

Corresponding author.
E-mail address: roseti@ing.uniroma2.it (C. Roseti).

http://dx.doi.org/10.1016/j.comnet.2016.11.002
1389-1286/ 2016 Elsevier B.V. All rights reserved.

ror control (both triggered by ACKs) and a reactive approach to


congestion.
On the other hand, both Internet applications and current networks are today completely transformed implying transport requirements highly evolved. From application side:
Applications generate frequent small objects to be transferred;
The number of connections increases as a function of the number of servers to be used;
Long transfers often have more exible delivery requirements
(for instance a long remote backup could run in background
slightly impacting on web smaller transfers) while in general
competing ows require a multi-priority management (fairness
metric must be reviewed).
From network side:
Diffusion of wireless access networks (i.e.[10,11]);
Terminal mobility and possible inter- or intra-network handover [12];
Dynamic network switching leveraging on the SDN paradigm
[13];
High storage capability in the network and cloud [14];
Very high capacity links [15];
High latency links [16].
The requirements above are expected to characterize the Future
Internet framework, as modeled by the fruitful adoption of the re-

A. Abdelsalam et al. / Computer Networks 112 (2017) 122143

cent technology advancements. Both technology evolution and the


new application characteristics make the TCP patching approach,
adopted so far, ineffective: solutions tailored for a specic scenario
can result inadequate to address a different one. For instance, a
modied TCP-based ow control aimed to achieve large congestion windows (cwnd) over large-delay satellite links, might cause
overload if applied over terrestrial-only links. This issue is particularly emphasized when adopting either dynamic switching technology (i.e. SDN) or mobile access, which lead to a high and unpredictable network dynamism. Nowadays, during a TCP-based connection, data path can change abruptly and transparently to the
end-points, accordingly to the recent 5G specications.
The proposed TCP Wave design is driven by the requirements
of both current trac characteristics and network evolution. In addition to inter-ows fairness, congestion avoidance, error recovery,
already addressed by traditional TCP, TCP Wave aims at guaranteeing also adaptation to bandwidth and/or delay changes, acceleration of small transfers, quick achievement of high throughput,
ecient handling of losses (from spurious losses to temporary link
disconnections) and so on.
Without loss of generality, this work aims to present all the
characteristics of the new transport protocol relying on a general
test environment reproducing the most challenging network dynamics, providing a suitable framework to validate the new transmission features. It is of paramount importance to remark once
again that the TCP Wave applicability is not limited to any specic
scenario.
2. Related work
Considering TCP NewReno as the reference, plenty of papers
have proposed to modify TCP congestion control schemes addressing one or several issues related to specic communication scenarios [6,7].
With the exception of optimized TCP versions installed on
PEP/Proxy agents [17], TCP NewReno [18] and its variants, ACKclocked and window-based, are running on most of Internet hosts.
However, standard TCP today is not substantially upgraded in actual end-systems because of two main problem categories:
1. Friendliness with standard TCP. Performance optimization for
TCP ows crossing links with large latency is often achieved
leveraging on aggressive strategies for the transmission rate
increase. The rationale is to contrast the effect of the large
Round Trip Time (RTT) over-inating transmission parameters
upon the reception of acknowledgments (ACKs), which trigger
new packet sending. When such ows share bottleneck bandwidth with other ows running standard TCP, an unfriendly
bandwidth re-partition could result. Denitively, a certain number of new TCP proposals dont guarantee adequate friendliness
with the standard TCP;
2. Design based on scenario-specic drivers. A large number of enhancements and/or new congestion control algorithms have
been designed to solve TCP impairments on specic links
and/or network segment. On the other hand, TCP needs to respect an end-to-end semantic and most of end-systems/userdevices can access to the network form different points (i.e.
laptop, tablet, smartphones), so that corresponding ows cross
completely different network segments over time. Therefore,
transport protocol implemented in the end-systems must be
able to work eciently over all the possible links.
A very high number of TCP variants fall in the rst category, as
those tailored for problems of bandwidth under-utilization problem over large bandwidth-delay link, due to the slow growth of
TCP congestion window (e.g., FAST [19], HSTCP [20], STCP [21],

123

Westwood [7]). In fact, most of these protocols deal with modifying the window growth function of TCP in a more scalable/rapid
fashion. Bandwidth utilization results improved, but a number
of issues remain as major challenges: friendliness with existing
TCP trac, fairness among ows experiencing different RTT, a fair
bandwidth sharing between long and short transfers.
One of the most successful upgrade of TCP is TCP Cubic [22].
It is currently used as default in the Linux kernels (2.6.19 and
above) and it is very important within the proposed work, since
it can be considered a relevant attempt to decouple cwnd increase
and the RTT. In fact, the cwnd growth mainly depends on the
time elapsed from the last congestion event, although still ACKtriggered. Undoubtedly, TCP Cubic demonstrated an improvement
on the achievement of the optimal transmission rate on a wide
set of channels (including long latency ones), but its clocking and
reactive nature (not proactive) to congestion events leave wellknown open issues: start up behavior is still slow in getting available bandwidth penalizing short transfers, which are dominant in
the current Internet; fairness among ows is achieved over a time
strongly dependent on the underlying link characteristics, leading
to unfair situations impairing short transfers. Last, but not least,
TCP Cubic design as well as other TCP versions, does not address
at all possible dynamic link changes during a connection establishment.
Recently, several new TCP proposals aimed to specically address end-to-end link variability and/or diversity and belong to the
second problem category.
OpenTCP variant [23], for instance, has been specically proposed for SDN-based Data Center networks. It utilizes information
provided by the underlying SDN network state (i.e., topology and
routing information) and trac information, in a sort of cross-layer
interactions. Depending on such feedback and congestion control
policies, dened by the network operator, OpenTCP decides on a
specic set of adaptations for standard TCP. In more details, a Congestion Control Agent (CCA) is installed as a module in the endsystems with the aim to manage update massages sent by SDN
controllers and modify TCP stack accordingly. TCP behavior can be
modied leveraging on two main options: by modifying feedback
provided to TCP sources, i.e. using ECN bits in the ACK segments,
or taking advantage from today extensible TCP implementations,
which allow triggering a new congestion control scheme. OpenTCP
relies on a computation cycle including data collection, feedback
generation and TCP adaptation. Timescale must be much slower
than experienced RTT to make network dynamics stable, while TCP
scheme changes are obviously on a per-ow basis. Extending its
application to non-SDN context would require much more effort
and modication in-network, with additional signaling protocols.
In [24], Multipath TCP (MPTCP) has been presented as a transport protocol conguration able to exploit multi-path capability
to manage transfers between end-systems pairs through different network interfaces and then following different routes. MPTCP
benets include better resource utilization, better throughput and
smoother reaction to failures. On the other hand, TCP behavior
over each path heritages issues already discussed for all the aforementioned TCP versions.
Last, but not least, TCP Noordwijk [25] can also be considered
as a specic-scenario transport protocol, since it has been originally designed to operate between satellite Performance Enhancing
Proxies (PEPs). Since TCP Wave inherits the basics sender architecture from TCP Noordwijk, its detailed description is provided in the
next section.
3. TCP Noordwijk: pioneer of the burst-based transmission
TCP Noordwijk [2527] is the rst TCP-based protocol that
overcomes the window-based transmission paradigm introducing

124

A. Abdelsalam et al. / Computer Networks 112 (2017) 122143

a burst-based transmission. In particular, BURST size replaces the


Congestion Window (cwnd) concept and an internal periodical
timer, namely TxTime, schedules burst transmissions, overcoming
the ACK-clocked timing used by traditional TCP. Bursts transmission is then completely decoupled from the ACK reception process, and must be carefully performed to avoid congestion or other
negative effects. Transmission in bursts allows the generation of
ACK trains, which carry useful information to eciently estimate
available bandwidth: RTT is a good meter of the current congestion, while ACK train dispersion tells about the bottleneck available capacity [28]. In particular, TCP Noordwijk performs an ACKbased capacity and congestion estimation (ACCE) function, which
monitors ACK ows and proactively estimates a set of related
statistics/information concerning the channel congestion status and
available capacity. TCP Noordwijk sender then uses such information to update both BURST and TxTime values according to two algorithms: Tracking Mode and Adjustement Mode. The former one
runs while RTT is estimated low and aims to achieve the maximum transmission rate. Adjustement Mode runs when RTT grows
above a given threshold ( ) in order to recover from a forthcoming congestion status.
TCP Noordwijk was specically designed to operate between
PEP agents in the frame of an European Space Agency (ESA) project
aimed to increase data transfer eciency over satellite links, in
particular (but not only) in relation to short data transfers when
Bandwidth on Demand (BoD) mechanisms are implemented. As
a result,it is able to optimize throughput for large latency links
and resiliency to delay variations caused by BoD mechanisms, using as reference the DVB-RCS standard [11,25]. Being a scenariodriven protocol, at the receiver side, the Selective Negative ACKnowledgments (SNACK) option, mandatory in the InteroperablePEP specication [17], was required for a correct protocol functioning. More details on TCP Noordwijk implementation and performance are available on [25], where description of the algorithms,
performance evaluation (also in comparison with other Transport
Protocols), fairness and friendliness are addressed.
4. TCP Wave description and implementation
TCP Wave follows the evolution started by the TCP Noordwijk,
but its target scenario is broader, aiming to become a valid alternative to all standard TCP versions in every broadband communication environment (including wired terrestrial-only links).
TCP Wave keeps the same overall protocol architecture and all
the basic burst-based transmission functions of TCP Noordwijk:
Implementation of a TxTime variable to schedule new bursts
transmission;
ACK-Based Capacity and Congestion Estimation (ACCE) functional block;
Basics of the Tracking Mode algorithm;
Basics of the Adjustment Mode algorithm.
On the other hand, TCP Wave proposes revised transmission algorithms and an enhanced method to process ACK trains tailored
to work also with cumulative and delayed ACKs, as envisaged by
default TCP receiver congurations.
Since the beginning, the protocol design was driven by the following concepts that are important to ensure its future deployment, effectiveness and validity on real networks:
no changes required at the receiver side (previously, as in e.g.,
[25], SACK option at the receiver was assumed);
design based on standard TCP implementations at the sender
side, without introducing custom low-level functions and callbacks (so that a possible porting to real Operating Systems, i.e.,
Linux is facilitated).

To this aim, NS-3 [29] simulation framework was selected for


the TCP Wave implementation, due to its network accurate models, exibility and adherence with real systems (in particular with
regard to TCP based sockets). Specically, the coding activity was
performed without changing the overall TCP state machine, but
only by inheriting and modifying standard NS-3 TCP API methods
(Socket Base class) for the sender (the receiver is unchanged):
TcpSocketBase::SendPendingData()
TcpSocketBase::Retransmit()
TcpSocketBase::Window()
TcpSocketBase::AvailableWindow()
TcpSocketBase::NewAck()
TcpSocketBase::DupAck()
On this basis, TCP Wave implements a set of enhancements,
aimed to overcome the main TCP Noordwijk limitations as summarized in Table 1. These enhancements are described in the following sub-sections, where both functionality and algorithms are
detailed include C++ pseudo-code reecting its implementation realized in NS-3.

4.1. Enhanced ow control


TCP Wave can operate with limited transmission buffer
(TxBuffer), adapting BURST size accordingly to the current free
buffer capacity. Flow control operations are implemented in the
following function:
TCPWave::CallSendPendingData ()
that internally manages and schedules the standard TCP API
TcpSocketBase::SendPendingData(withAck)
This TCP Wave function (as well as all the other presented later
on) properly handles traditional TCP variables and functions, while
building the new burst-based transmission. In this specic case,
the CallSendPendingData() is rescheduled every TxTime and
it is in charge to compute the proper cwnd value, allowing the immediate transmission of a new BURST of packets. In general, the
current cwnd is increased by BURST in order to allow the generation of a new burst. Flow control is mainly in charge of checking if
Receive Window (rwnd) (advertised in the ACK packets) allows the
computed cwnd and consequentially the transmission of a further
burst with size equal to BURST. Specically, the applied rule is:
if (cwnd > rwnd) cwnd=rwnd;
In addition, TCP Wave performs also a check of the actual
amount of application data, not yet transmitted, stored in the
transmission buffer (TxBuffer). In fact, resulting cwnd could exceed
TxBuffer. In this case, all the data already present in the TxBuffer
are sent in the rst burst available, while the remaining amount
of data allowed by the BURST value will be transmitted later, not
respecting the TxTimer (fresh application data is transmitted as
soon as some segments are removed from the transmission buffer
upon the reception of the corresponding ACKs). Such a process, as
a function of the underlying TCP state machine, is outside of the
TCP Wave ow control, then compromising the transmission burst
structure, and consequentially the ACK train dynamics. To avoid
this misbehavior, ow control has been enhanced to perform a further check on the computed cwnd:
cwnd = min (txBuffer.size(), cwnd);
In practice, the actual BURST size is possibly decreased in accordance to the amount of data not yet transmitted currently present
in the transmission buffer. Compared to this Flow Control, TCP Noordwijk directly operated on BURST parameter and did not perform
any control on the effective availability of fresh data on the transmission buffer.

A. Abdelsalam et al. / Computer Networks 112 (2017) 122143

125

Table 1
TCP Noordwijk limitations tackled by TCP Wave.
TCP Noordwijk function

Weakness description

Flow Control

Assumption of transmission buffers large enough to always include fresh data on the next scheduled BURST. Flow
Control is receiver-driven only.
Tested and validated with Selective ACK (SACK) option enabled only, not interoperable with common Operating
Systems (O.S.s).
Loss recovery through SACK notication only.
RTO timer management inherited from standard TCP, not suitable for link disconnections.
Triggering of the Adjustment Mode algorithm leverages on the raw RTT measurements, then affected by possible
inconsistent samples.
Tailored to transmit at the maximum bottleneck speed irrespective of presence of competing ows.
Tailored to perform an abrupt rate decrease waiting for the congestion reduction. Its duration its unpredictable and
depends on the nature of the competing trac.
Stability Factor (SF) denes the timing for transmission parameter changes. Potential loss of responsiveness to sudden
changes.

ACK management
Fast Retransmit
Retransmission Time Out (RTO)
Switching between rate control modes
Tracking mode algorithm
Adjustment mode algorithm
Stability Factor method

4.2. Management of cumulative and delayed ACKs


TCP Wave adds some new features in order to eciently operate with cumulative and delayed ACKs, which are foreseen by most
of the Operating Systems (OSs), running on common servers and
user devices.
4.2.1. Split of cumulative ACK
With the aim to identify correctly each ACK received in trains,
TCP Wave rst re-creates a tight one-to-one correspondence between packet and its corresponding ACK. At this scope, the following function
TcpWave::NewAck(SequenceNumber32 const& rAck)
performs the splitting of cumulative ACKs (not duplicated) into
multiple individual ACKs, each referred to a single TCP segment.
Such individual ACKs are then one-by-one passed and processed by the ACCE component:
for (uint16 i=1; i <= segmentAcked; ++i)
{
SequenceNumber32 ack =
txBuffer.HeadSequence()
+ (segmentSize +1);
Time rtt = now - GetTransmitTime(ack);

ACCE (segmentSize, rtt, now);


}

AckTrainDispersion = LastAckTime - FirstAckTime


Then, to compute the ACK dispersion, it is needed to divide the
above train dispersion time for the number of equally spaced intervals among ACKs, which is equal to (BURST 1 ):
AckDispersion = AckTrainDispersion /(BURST-1)
Finally, although two different receiver congurations should
lead to the same AckDispersion (assumed as an average value), the
actual result can be different as the rst ACK is delayed of a DelAck
interval. To take into account of such a time dynamic, the number
of inter-spaced intervals to consider for the calculation of the AckDispersion value can be dened as (BURST DelAckCount ):
AckDispersion = AckTrainDispersion /(BURST-DelAckCount)
4.3. Enhanced Fast Retransmit (FR) algorithm
As previously described, TCP Wave must work with standard receivers and ACK options. Therefore, packet loss recovery is supported in case of basic TCP receiver conguration, where duplicated ACKs are sent in case of out of sequence reception of packets. Accordingly, TCP Wave implements a function to handle duplicated ACKs (dupACKs) and schedule retransmissions. Similarly to
the standard TCP, upon reception of the third dupACK, the corresponding segment is immediately retransmitted. Then, as in TCP
Noordwijk, BURST size for the next scheduled burst is reduced of
the number of retranmitted data. Such a control procedure is implemented in the function:
TcpWave::NewAck (SequenceNumber32 const& rAck)

4.2.2. ACK train dispersion


Computation of consistent ACK dispersion samples (meant as
the average time between two consecutive ACKs) is of paramount
importance, since these values contribute to update the actual TxTime parameter (see sub-Section 4.7). In an ideal case, with the
delayed ACK option disabled (DelAckCount = 0) in the receiver and
no losses, every TCP segment generates an ACK, and ACK reception
time interval during a train reects dispersion due to bottleneck
link.
Since in general DelAckCount is not zero (for instance in real
O.S.s and as default in NS-3 it is equal to 2), the receiver attempts
to transmit one ACK every more than one segment (if received
within a pre-set DelAck interval). Therefore, since ACCE function
must work on individual ACKs split as described before, timing is
quite different from the case of not delayed ACKs. A logical representation is shown in Fig. 1.
To make the estimation more robust, taking into account also
the articial split of ACKs when DelAckCount is not 0, it is better to
estimate the whole train of ACK dispersion rst, easily calculating
it as follows:

Since the burst-based transmission usually causes a very high


number of dupACKs during the retransmission phase, the cumulative NewAck, generated by the successful reception of a retransmitted packet most likely acknowledges a very high quantity of
data. Consequentially, the ACK splitting function, presented above,
generates several trains of individual ACKs passed at the ACCE
function very close in time and providing a similar RTT inated by
the retransmission time. In such conditions, ACK Train Dispersion
calculation can be affected by inconsistent values. A sketch of the
various implications on the ACCE statistics calculations, depending
on the start of dupACKs generation, is depicted in Fig. 2.
In the modeled example, the Nth ACK train includes the sequence number of the lost packet, in particular of the second
packet of the train. Then, rst ACK is regularly received within the
expected schedule, while all the other ACKs will be carried by the
cumulative one triggered by the correct reception of the retrasmitted packet, about one RTT later. In the meanwhile, a large number
of dupACKs is received. In these circumstances, Nth Ack Train Dispersion is much higher than the current RTT value. To opposite,
following ACK trains are entirely composed of individual ACKs

126

A. Abdelsalam et al. / Computer Networks 112 (2017) 122143

Fig. 1. Reception timing of delayed-cumulative ACKs.

Fig. 2. Cumulative ACK reception process.

derived from the same cumulative ACK, so that they lead to the
computation of an ACK train dispersion close to 0. For all these
reasons, TCP Wave introduces a set of lters for both RTT and
Ack Train Dispersion calculations. As far as the RTT is concerned,
the procedure of splitting the cumulative ACKs has been enhanced
with the following lter:
if (bytesAcked > 2 * segmentSize)
Time RTT= avgRTT;
Basically, TCP Wave sender performs only a single ACCE update
every cumulative ACK.
In addition, the following lter is implemented to lter altered
Ack Train Dispersion sample:
if ((LastAckTime - FirstAckTime) > avgRTT
&& (LastAckTime - FirstAckTime) == 0)
ackTrainDispersion=minTrainDispersion;
This algorithm discards ACK train dispersion samples higher
than measured RTT or equal to 0, which are expected during retransmission phases.
4.4. Enhanced Retransmission Time Out (RTO) mechanism
Retransmission Time Out (RTO) indicates how long TCP senders
must wait for an ACK before performing a retransmission action
and it works in a tight combination with Fast Retransmit (FR) algorithm of TCP. RTO is usually set proportionally to RTT measurements, which are computed once per burst in TCP Wave. When
a lost packet is retransmitted through FR algorithm (Section 4.3),
the resulting RTT value is inated because RTT is measured against
the original packet transmission. Obviously, in this case the RTT increase is not a symptom of congestion. Differently from standard
TCP, all the RTT measurements including those calculated during
retransmissions, are sent to the underlying RTT estimator, which

calculates the round-trip time variation (RTTVAR) and sets the RTO
value accordingly. This expedient ensures that the estimated RTO
value is always consistent with the RTT value seen by TCP Wave,
also during retransmission, avoiding unnecessary and unwanted
RTO expirations. The second aspect tackled by TCP Wave for the
RTO management is related to the actions to perform upon RTO
expiration (RTO recovery). Without receivers feedback for a time
that exceeds the estimated RTO, the sender assumes that all segments that were previously sent, but not yet acknowledged, are
lost. Thus, data recovery is performed starting from the rst unacknowledged segment until the last segment that has been sent
when the RTO is expired. Because RTO usually indicates a serious problem in the network, either severe congestion or even
temporary disconnection, TCP Wave sender behavior during the
RTO recovery is quite conservative: the sender resets the BURST
size and the TxTime to the initial values BURST0 and TxTime0 respectively, assuming that network conditions are not well known
yet. Then transmission restarts in bursts from the latest Acknowledged packet before the RTO expiration. If cumulative ACKs related to packets belonging to multiple bursts are received during
RTO recovery, TCP Wave sender accounts them for jumping in the
sequence number space, and to update ACK-based statistics only
once. This produces gaps between the retransmitted bursts, reducing the amount of data unnecessarily retransmitted. In the meanwhile, BURST size and TxTime are still dynamically updated upon
the reception of regular ACK trains.
4.5. Switching between tracking and adjustment mode
In order to avoid effect of unexpected RTT spikes due to several network dynamics (i.e. processing glitches, link layer retransmissions, etc.), TCP Wave considers an average RTT value to compute RTT. In addition, an Exponentially Weighted Moving Average

A. Abdelsalam et al. / Computer Networks 112 (2017) 122143

(EDWA) lter with a lter parameter k is considered:

AvgRT Tk = k AvgRT Tk1 + (1 k ) RT Tk


k indicates the number of received ACK train and RTT is referred to
the rst packet of the target burst. The value of the lter parameter
k determines the degree of ltering. Small values correspond to
a responsive lter (fast reaction to changes), while larger values
increase lter stability. TCP Wave dynamically computes k on the
basis of the following formula:

k =

RT Tk minRT Tk
RT Tk

Thus, the lter is very reactive when RTT is close to the absolute minimum measured value minRTTk , updated at each received
ACK train. In other words, smaller RTT samples greatly inuences
the new average RTT estimation, while larger RTT samples have a
minor effect. Denitively, RTT, used as condition to switch between Tracking and mode, is computed as follows:

RT Tk = avgRT Tk minRT Tk


This makes RTT tracking more stable avoiding unnecessary triggering of the Adjustment algorithm. Given [s] as a parameter
that expresses the tolerated level of congestion in terms of RTT
increase, while RTT < , TCP Wave remains in Tracking mode,
otherwise it switches to Adjustment mode.
4.6. Tracking mode and Wave effect
The Tracking mode is the main protocol algorithm and it has as
primary objective to achieve the optimum transmission rate, taking
into account both the bottleneck bandwidth and the presence of
competing ows. It leverages on a continuous monitoring of both
ACK train dispersion and RTT variations, as provided by the ACCE
component, in order to properly adjust transmission parameters.
To this aim, a new equation-based TxTime computation is proposed, while keeping xed the BURST size to its default value. Basically, TxTime is updated on the reception of any ACK train as a
linear combination of:
AckTrainDispersion, which assesses the capacity of the overall
end-to-end system to process a single burst. It represents the
time interval among bursts to achieve the maximum rate allowed by the bottleneck link independently from the presence
of concurrent trac;
RT T = avgRT T minRT T , which is proportional to the forthcoming congestion. The higher is RTT, the heavier is the current network load.
Therefore, the following equation has been implemented:

T xT ime = AckT rainDispersion + 0.5 RT T


The 0.5 factor is justied because RTT increase is of course
due to the joint effect of the target ow and all the competing
ows. Then, the rate reduction is accounted only for one half to
the target ow. This action has a threefold goal: to guarantee fairness among TCP Wave ows, to be friendly towards non-TCP Wave
ows and to guarantee an optimum channel utilization. In addition, a further control in the code aims to limit the maximum eligible TxTime to its initial value TxTime0 , to guarantee a minimum
transmission rate in accordance to the pre-set default values (see
sub-Section 4.9):

T xT ime = min(T xT imer, T xT ime0 )


Finally, the actual transmission rate computed in Tracking mode
is given by:

T xRate =

BURST0
T xT ime

127

As a result, TCP Wave shows a new response coming from the


dynamic variation of the above described TxTime computation. As
shown in the simulation part of the paper, the variable scheduling based on TxTime shapes a wave-like trend for burst transmission (slowing down rate with the increase of burst distance, and
gain a rate acceleration decreasing the distance). Denitively what
will be called the wave effect is the straightforward effect of the
attempt to constantly achieve an equilibrium between congestion
avoidance and the maximization of the available bandwidth. With
such a scheme, until RTT is below a pre-dened threshold, TCP
Wave remains in Tracking mode.
On the contrary, the Adjustment algorithm is now triggered
with the aim to reset transmission parameters. Note that since TCP
Wave aims to adapt its rate in order to prevent congestion, the
triggering of the Adjustment algorithm represents an investigation
on possible changes on the networks (link handover, bandwidth
variations, routing changes, or simply presence of uncontrolled
ows such as UDP streaming), instead of a simple congestion recovery strategy.
4.7. Adjustment mode
When triggered (RTT > ), the adjustment algorithm performs a reset of the transmission variables and then immediately
returning in Tracking mode. The rationale is to allow an analysis
from scratch of network conditions without the inuence of both
the minimum RTT and the average RTT values, previously measured over the running connection. In fact, such values may not
be consistent any more, as consequence of some physical or network changes. In particular, two main situations could be in place
when entering in adjustment mode:
1. there is a persistent congestion;
2. something is changed in the network; e.g. handover to a link
with an higher delay, increasing of the access delay due to
some Bandwidth on Demand MAC protocol, routing changes,
etc.
In order to address the latter situation without compromising
the former, TCP Wave envisages a reset of the minRTT and the forcing of the avgRTT to the maxRTT value in order to avoid negative
calculations of the RTT. This procedure allows TCP Wave in the
case (2) to update minRTT when receiving the next ACK train with
a consistent value concerning the new possible network conguration. To opposite, in case (1), in presence of a persistent congestion
state, avgRTT is expected to continuously grow proportionally to
the congestion level, then returning into Tracking mode with RTT
= 0 (then with a even decreasing rate) and periodically meeting
conditions to newly trigger to the Adjustment mode. The higher
is the congestion level, the more frequent is the triggering of the
Adjusting mode algorithm.
Denitively, the new Adjustment mode algorithm computes a
burst size equal to the minimum burst size ( 3 segments to preserve burst structure) and TxTime is forced to TxTime0 , to achieve
the minimum allowed transmission rate with the aim to reduce
congestion state.
4.8. Stability factor review
TCP Noordwijk leveraged on a Stability Factor (SF) parameter to
regulate how frequent to apply variations in the transmission variables: BURST and TxTime. The rationale behind that was to wait
for effects of each re-tuning of transmission activity before making further modications. In practice, SF indicates the number of
ACK train to skip before checking channel parameters again. In fact,
since TCP Noordwijk alternates a transmission at the maximum
allowed rate during Tracking mode, with a strong rate reduction

128

A. Abdelsalam et al. / Computer Networks 112 (2017) 122143

for each triggering of the Adjustment algorithm, the changes upon


every received ACK train in an increasing RTT situation (above
threshold) could lead to a remarkable overall rate reduction. For
this reason, TCP Noordwijk usually uses an SF value close to the
experienced RTT. On one hand, this strategy allows to avoid unnecessary rate reduction before feeling new link state but, on the
other hand, it makes rate control less reactive to fast changes in
the network. In this regards, TCP Wave can take advantage from
the new Tracking algorithm, which aims to prevent the triggering
of the Adjustment algorithm, being able to correct the transmission
rate directly in Tracking mode. Therefore, with the new Tracking
mode, TCP Wave ignores SF and it actually computes and applies
changes on transmission parameters upon every ACK train reception. To opposite, TCP Wave sets the SF only during Adjustment
mode to allow effect of the burst reduction to be effective. In particular, SF is computed as the number of ACK trains received over
an average RTT period:

SF =

avgRT T
ACKT rainDispersion

Therefore, the actual time duration of the adjustment phase can


be obtained as:

Max(T xT ime0 , avgRT T )


4.9. TCP Wave parameters
TCP-Wave relies on four parameters that inuence the overall transmission process: the initial burst size, BURST0 , the initial
transmission time (spacing between consecutive bursts), TxTime0 ,
the minimum allowed burst size, BURSTmin , and the threshold, used to manage switching between Tracking and Adjustment
mode. These values can be chosen in order to matching requirements of either the target application or the target communication
link.
First, BURST0 is the reference burst size used in the Tracking
mode. Such a value of course determines the size of the ACK trains.
BURST0 can not be < 3 to guarantee a minimum burst train structure. It also cannot be too small in general, to avoid too short ACK
trains which are less suitable to offer robust channel estimations,
and it cannot be too big, since the bigger is BURST0 and the higher
will be the computed TxTime (depending in turn on the ACK train
dispersion). Values ranging from 10 to 20 TCP segments have been
proven to support a ACK train structure providing reliable values
for ACK dispersion estimation.
TxTime0 contributes together with BURST0 to achieve the initial
transmission rate before the reception of the rst ACK trains, or
after a resetting of the parameters (i.e. after an RTO expiration):

RateINIT =

BURST0
T xT ime0

In addition, TxTime0 is used as TxTime value in the Adjustment


Mode. Therefore, its value must be set big enough to allow a prudent start up and to not limit rate reduction in the Adjustment
Mode. Typical values are in the range of 0.5 1 s, which correspond to highest physical RTT experienced in current networks (i.e.
worst case if using GEO satellite links).
BURSTmin impacts on the minimum allowed rate during the Adjustment Mode, which is given by:

RateMIN =

BURSTmin
T xT ime0

As previously discussed, BURSTmin can not be smaller than 3


TCP segments in order to preserve the ACK train structure.
Finally, is a time-constant indicating the tolerated RTT increase. The higher is , the longer is the expected period in Tracking mode. This has a twofold effect:

1. Rate correction in Tracking mode lies on a larger range of values, because:

BURST0
BURST0
Ratetrack
T xT ime +
T xT ime
This also implies a better capability to achieve a fair and stable bandwidth sharing in case of multiple TCP Wave ows;
2. Triggering later the Adjustment mode, where RTTmin is reset,
TCP Wave is slower in experiencing possible network changes
and in general has a gentler interaction with competing ows
(e.g. TCP NewReno ows).
Typical values for are in the range 0.1 0.3 s.
5. Simulation setup
TCP Wave has been implemented on the Network Simulator NS3 [29]. A new module tcp-wave.cc has been added at the opensource software package allowing the conguration of a TCP Wave
sender object to be mounted on a generic sender node for transporting application trac. The reference conguration of the simulation scenario is depicted in Fig. 3. A variable number of source
nodes are connected to the Router 1 through point-to-point links,
characterized by a low latency. Each source nodes generates a congurable amount of raw data over reliable connections exploiting
either TCP or TCP Wave services. Then, Router 1 is connected to
the Router 2 through the bottleneck link, which is shared among
all the running data ows. Finally, data ows generated are terminated at separated Sink agents, each directly connected to Router 2.
If not explicitly reported , link parameters will be those included
in Fig. 3.
As far as the initial TCP Wave parameters are concerned, the
following default values are considered:
1.
2.
3.
4.
5.

MT U = 400 bytes
BURST0 = 10 T CP packets;
T XT ime0 = 0.5 s;
BURSTmin = 3 T CP packets;
= 0.15 s;

6. Protocol validation and performance assessment


TCP Wave start up phase envisages a burst transmission (black
lines in Fig. 4) using default parameters: BURST0 and TXTimer0 .
A rst update of the transmission is possible upon the complete
reception of the rst ACK train (red points in Fig. 4). Therefore,
showing the rst two seconds of a TCP Wave transfer, the rst
two TxTime rescheduling are performed using TXTimer0 value: note
that after transmission of the second burst, the rst ACK train is
not fully received and then TxTime can not be updated. From third
burst and forth, TxTime is set accordingly to the estimated ACK
train dispersion, allowing to achieve the maximum rate, as demonstrated by the subsequent reception of a continuous ACK ow. In
other words, just after the receiver processes the last segment of a
burst and sends an ACK, the rst segment of the next burst arrives
and triggers a further ACK. As a consequence, the ACK ow arrives
at the sender without time interruptions.
Since there are not competing ows, TCP Wave allows a stable burst transmission at the maximum rate of 2 Mbit/s as soon
as Tracking algorithm is triggered at 1.2 s. It is important to remark that the resulting slope of the burst transmission curve is
proportional to the actual transmission rate. The RTT always oscillates between the propagation delay and propagation delay plus
AckTrainDispersion. In Fig. 4, RTT is represented by the difference
between burst transmission time and initial reception of the corresponding ACK train.

A. Abdelsalam et al. / Computer Networks 112 (2017) 122143

129

Fig. 3. Conguration of the simulation scenario.

Fig. 4. TCP Wave start up behavior.

6.1. Tracking mode and Wave effect


TCP Wave also achieves a good degree of fairness among competing ows. Fairness is managed through the new Tracking algorithm, which inates TxTime, set to the computed AckTrainDispersion, using current RTT. Therefore, as shown in Fig. 5, each TCP
Wave ow experiences in practice the presence of the other one
by continuously modify actual TxTime in use, originating a sort of
oscillation in the burst transmission timing, herein dened as wave
effect.
In addition to long-term fairness, the protocol also allows to efciently perform short data transfer. In fact, the existing connection is quickly reacting to new channel conditions, allowing the
initial transmission of the new connection at RateINIT (critical for
short-objects), without impacting on transmission continuity. Afterwards, the connection just started and the existing one converge to a new equilibrium condition. This characteristic was already widely discussed in [25] and remain valid in this new TCP
Wave implementation. By analyzing simulation output, it is possible to distinguish three different phases:

1. in the rst 10 s, a single TCP Wave ow is running achieving


the full rate of 2 Mbit/s (from 1.2 s to 10 s about 5500 packets
of 400 bytes are transmitted);
2. In the time interval 1040 s, the burst sequence number trends
of the two ows go in parallel, demonstrating a similar transmission rate. In addition, the slopes of the curves are about one
half of the one observed when a single ow was running. This
means that both ows run at around 1 Mbit/s;
3. At 40 s, a ow stops and the remaining one suddenly gains the
full capacity.

Fig. 6 provides a zoom of Fig. 5 in the range 1216 s, for the


rst ow, revealing the effect of the new algorithms at the occurrence of competition. Basically, each ow senses the other one
by mean of RTT increases due to the processing of simultaneous
bursts arriving at the bottleneck link. Upon RTT increases, each
ow increases TxTime, and then the spacing among consecutive
bursts. In turn, TxTime ination leads a subsequent rate and RTT
decrease. Such periodical RTT increases and decreases denitively
produce rate oscillations around the fair rate value equal in the

130

A. Abdelsalam et al. / Computer Networks 112 (2017) 122143

Fig. 5. TCP Wave fairness.

Fig. 6. Details on Wave effect, associated to the smooth increase and decrease of TxTime as function of the sequence number.

case to half capacity, making a burst sequence trend similar to a


sea Wave, which gives the name to the protocol.
Extracting numerical values from simulation output series, in
the time interval 1040 s ow 1 achieves an average transmission
rate of 943 kbit/s a, while ow 2 of about 913 kbit/s, for a total
throughput of 1856 kbit/s (over the 20 0 0 kbit/s available). In conclusion, fairness is granted while capacity utilization ineciency is
limited. However, this drawback in the overall capacity utilization
is compensated when more than two long ows run simultaneously, as demonstrated later on.
Finally, the wave effect allows keeping experienced RTT always
close to its minimum value, although some spikes are due to the
handling of simultaneous burst in the system queue. As shown
in Fig. 7, RTT spikes for ow 1 are limited to an upper-bound
value depending on the number of simultaneous ows and on the
BURST0 value. Then, this maximum RTT converges to a value estimable a priori.

6.2. Channel utilization and multi-ow fairness


To have an exhaustive assessment of both TCP Wave fairness
and capacity utilization, a more complex simulation scenario has
been set up. Four long transfers over TCP Wave connections start
and stop at different times. Considering a total simulation duration
(D) of 200 s, the start and stop time for the ith (with i = 0,1,2,3)
ow are scheduled as follows:
Start at: i 30;
Stop at: D i 30.
In this way, the competition status continuously changes along
the simulation, with a number of simultaneous ows ranging from
1 to 4. Fig. 8 shows burst transmission evolution (sequence number). Until 30 s, the rst ow runs alone. It achieves the maximum
throughput thanks to the capability of the Tracking algorithm to
compute the optimal TxTime.
After 30 s, a second TCP Wave ow starts. The slopes of the
two curves converge at a similar inclination, which is half of the
slope observed for a single ow. Average throughput is respec-

A. Abdelsalam et al. / Computer Networks 112 (2017) 122143

131

Fig. 7. RTT variations.

Fig. 8. TCP Wave fairness in a multi-ow conguration.

Table 2
Transmission rate summary (kbit/s).

Flow
Flow
Flow
Flow
Total

1
2
3
4

Int. 1

Int. 2

Int. 3

Int. 4

Int. 5

Int. 6

Int. 7

1991

1991

955
832

1789

680
697
610

1987

546
509
498
447
20 0 0

668
674
615

1957

887
868

1755

1930

1930

tively 955 and 842 kbit/s as reported in Table 2. Total throughput


is 1798 kbit/s, a bit lower than the available capacity, as already
observed in the previous simulation test. The utilization comes
back to be optimal when a third ow joins the previous two (after
60 s). The achieved total average throughput is about 1987 kbit/s.
The negligible difference from the maximum theoretical value of
20 0 0 kbit/s is due to the initial blind phase of the entering ow
that starts its transmission using default transmission parameters.
With three competing ows also fairness improves, since average
transmission rate measured for the competing ows is closer. The

achievement of optimal performance is consolidated in the case of


four competing ows (in the range of 90120 s). Maximum utilization and a fair bandwidth sharing are achieved. From 120 s, one
by one ows stop at regular intervals of 30 s. Results demonstrate
stability of TCP Wave control protocol, which allows to quickly readapt transmission rate on the basis of the actual number of running ows.
This simulation outcome also demonstrates TCP Wave capability to avoid intra-protocol starvation events, i.e., not blocking/impairing new connections entering. In fact, the proactive rate
tracking method allows both old and new connections to feel
each other and to fairly adjust the rate accordingly.
RTT was monitored during the multi-ow simulation, being a
meter of the induced congestion. As shown in Fig. 9, RTT cyclically returns to its minimum value irrespective of the number of
competing ows. In parallel, periodical RTT spikes are generated
upon the arrival at the bottleneck of simultaneous bursts. This result demonstrates how TCP Wave allows a complete control of the
experienced delay at the end-points, which is kept below a target estimable value. The maximum amplitude of peaks is only due

132

A. Abdelsalam et al. / Computer Networks 112 (2017) 122143

Fig. 9. RTT and average RTT in a multi-ow conguration.

the number of simultaneous ows, but is not diverging over time.


Knowing the maximum number of long ows and default TCP
Wave parameters, it is possible to predict the maximum RTT value
and accordingly to tailor the system or, vice-versa, once known
characteristics of the target communication environment, it is possible to nd the better conguration for TCP Wave. By the way, the
achieved RTT value corresponds to a storage capability of several
hundreds of Kbytes in the bottleneck queue. Such a requirement is
easily and abundantly matched in most of the current broadband
networks. Again, in case of small buffers and extremely high number of TCP Wave competing ows, scalability is guaranteed by RTO
mechanism, which is most likely triggered up on signicant overows (see RTO-related behaviour in Section 6.5.2).
6.3. Reaction to path/bandwidth changes
Further tests have been performed in scenarios where dynamic
switching capability can lead to abrupt link changes during a TCP
transfer. Smart switches (i.e. SDN-enabled) can suddenly change
path leveraging on ad-hoc policies. Therefore, an end-to-end transport protocol for future Internet scenarios must face unexpected
changes in terms of both latency and bandwidth. In order to assess TCP Wave capability in covering such scenarios, a rst test
addresses two consecutive bandwidth changes during a TCP Wave
transfer. Initially, bottleneck bandwidth is set at 2 Mbit/s as in the
previous tests. At 5 s, the bandwidth is increased to 10 Mbit/s; at
10 s, the bandwidth is decreased to 5 Mbit/s.
During the whole connection, TCP Wave must face three different bandwidth constraints. This is a good scenario to highlight
the capability of the Tracking algorithm to follow available bandwidth using measurements of ACK train dispersion. Fig. 10 gives an
overview of the overall protocol behavior: transmission rate, proportional to the line inclination, is adjusted upon every bandwidth
change to the maximum allowed capacity. This can be easily conrmed by deriving average rate from the overall number of segments transmitted in the various time intervals.
In order to analyze in detail protocol dynamics during bandwidth changes, time intervals around bandwidth change events
have been zoomed in Figs. 11 and 12.
At 5 s, the available bandwidth is increased from 2 Mbit/s to
10 Mbit/s. TCP Wave is able to quickly feel this variation when receiving the rst ACK trains related to bursts sent over the 10 Mbit/s

link. As shown in Fig. 11, TxTime is updated about 0.5 s after the
bandwidth change, a time interval corresponding to the minimum
RTT envisaged in the congured simulation scenario. Because of
the bandwidth increases, the only apparent effect on transmission
dynamic is a sudden shortening of the TxTime, while burst size is
kept constant since the protocol remains in Tracking mode.
When halving bandwidth at 10 s, instead, transmission with
the old settings causes a temporary but inevitable RTT increase.
TCP Wave suddenly reacts to RTT increase by adjusting actual TxTime according to the tracking mode algorithm, which depends on
RTT. When RTT is brought back to its minimum value, transmission continues by using the optimal TxTime computed in the meanwhile, considering new measured values of ACK train dispersion.
Fig. 13 reports the RTT variations over the simulation time.
Bandwidth increase at 5 s does not lead any ination of the RTT,
while the bandwidth reduction at 10 s causes a temporary increase
of RTT, which doubles its value for a time interval of about 1 s (2
physical RTTs). Afterwards, TCP Wave is able to both achieve new
optimal transmission rate and empty the bottleneck buffer, bringing back RTT to its minimum value.
To cover all possible scenarios, an additional simulation setup
envisages a simultaneous change of both bandwidth and delay.
This scenario is very challenging for any transport protocol, since
it causes a fully changing of any time reference. Specically, at 5 s,
bandwidth changes from 2 to 10 Mbit/s and physical delay of the
bottleneck link increases of 50 ms (RTT is expected to increase of
100 ms, +20%). The result in terms of transmission dynamic, shown
in Fig. 14, is similar to the one experienced with only bandwidth
change. TCP Wave is able to adjust its transmission parameters to
match the new available capacity in the bottleneck, being resilient
to the RTT change.
The link delay increase makes the reception of ACK trains,
bringing new channel information, delayed. During the rst RTT
after delay-bandwidth change, a temporary glitch slightly increases
RTT above the new minimum value (see Fig. 15).
6.4. Flow control
In general, Transmit/Receive (Tx-Rx) buffer size should be at
least larger than bandwidth-delay product to guarantee a full exploitation of the available capacity. In case this condition is not respected, meaningful consequences may affect dynamic scenarios,

A. Abdelsalam et al. / Computer Networks 112 (2017) 122143

Fig. 10. TCP Wave behavior against bandwidth changes.

Fig. 11. TCP Wave behavior against bandwidth increase.

Fig. 12. TCP Wave behavior against bandwidth decrease.

133

134

A. Abdelsalam et al. / Computer Networks 112 (2017) 122143

Fig. 13. RTT variations during bandwidth changes.

Fig. 14. TCP Wave behavior against a combined bandwidth-delay change.

where network characteristics can change during a connection lifetime and end-system settings can not be intrinsically optimized for
all the network characteristics.
Rate regulation according to end-point buffering capability is in
charge of the Flow Control mechanism. TCP Wave envisages a Flow
Control scheme acting on the actual burst size. The burst size, as
computed by the rate control component, is corrected on the basis
of the actual amount of fresh data in the TxBuffer (data not yet
transmitted in previous bursts). If the amount of fresh data is
smaller than the computed BURST value, the denitive BURST size
is reduced accordingly or even set to 0 waiting for reception of
ACK trains, which allow to cancel corresponding ACKed segment
from TxBuffer, leaving space for new application data.
In order to validate the effectiveness of the TCP Wave Flow Control, a critical scenario has been reproduced setting overall capacity to 4 Mbit/s. With 4 Mbit/s and an RTT of about 500 ms, the
achievement of the maximum throughput requires an amount of
data in ight (and buffered in the TxBuffer) at any time equal to
about 250 kBytes. Then, when using TX-RX buffers smaller than

250 kBytes, there is not enough buffered data to continuously


transmit at the maximum rate. Fig. 16 shows the actual BURST size
computed when TxBuffer is equal to 130 kBytes. There is a clear
alternation of transmissions of the maximum burst (BURST0 = 10)
with periods with burst forced to 0 (no transmission) in order
to follow buffer occupancy trend. Between maximum burst transmission and no-transmission events (BURST = 0, with signicant
transmission ineciency), a gradual reduction of the burst size following the occupancy buffer dynamic can be observed. It can be
noted that according to ow control and application data availability (within the TxBuffer), smaller bursts than BURSTmin can be sent
(e.g., 1 packet), but in this case corresponding ACKs do not contribute to the sender statistics update.
Increasing TxBuffer to 210 kBytes (closer to bandwidth-delay
product), BURST = 0 cases are greatly reduced. As shown in Fig. 17,
a smaller reduction of the burst in the 59 segment range is
enough to not exceed TxBuffer capacity.

A. Abdelsalam et al. / Computer Networks 112 (2017) 122143

135

Fig. 15. RTT variation during bandwidth-delay change.

Fig. 16. Burst size computation with TxBuffer = 130 kBytes.

6.5. Loss recovery


TCP Wave detects packet losses similarly to traditional TCP
method, using a Retransmission Time Out (RTO) and 3 dupACKs as
loss notications. This guarantees a full interoperability with standard TCP receivers.
6.5.1. Test on fast retransmission (FR)
Upon the reception of the third dupACK, TCP Wave immediately
schedules the re-transmission of the lost packet without modifying
the current transmission rate (this means that sporadic packet loss
are not indication of congestion). In fact, the only action on data
ow is a reduction of next BURST size in the amount of the number of re-transmitted TCP packets. The rationale is to maintain constant the actual transmission rate. To validate such a method, we
added to the reference simulation scenario a random Packet Loss
Rate (PLR) generator, attached to the bottleneck link. A P LR = 103
value has been considered in the simulation in order to reproduce
frequent spurious losses. Fig. 18 shows achieved result with red

points representing retransmissions. Burst transmission scheduling


does not present any apparent variation upon retransmissions, indicating a substantial resilience in maintaining maximum transmission rate.
Fig. 19 shows RTT ination in correspondence to retransmissions. Using dupACKs as loss notication, every lost packet
within in-ight bursts can be recovered in an RTT. Accordingly,
Fig. 19 shows many RTT spikes at about twice the propagation RTT.
The few spikes with values higher than 2 RTT are due to multiple
losses within in-ight bursts at a given time.
6.5.2. Test on Retransmission Time Out (RTO)
The proactive burst transmission in TCP Wave maximizes the
possibility to have a continuous ow of either ACKs or dupACKs in
the leaky links. This allows to exploit a sucient number of dupACKs to recover multiple losses avoiding as much as possible RTO
expiration. In general, TCP Wave design aims to anticipate strong
congestion and then network troubles leading to RTO. Nevertheless, link outages can still happen due to physical problems or link-

136

A. Abdelsalam et al. / Computer Networks 112 (2017) 122143

Fig. 17. Burst size computation with TxBuffer = 210 kBytes.

Fig. 18. Re-transmission scheduling with PLR = 103 .

related operations and in these cases TCP Wave must face an RTO
occurrence. This scenario has been reproduced by forcing a 2 s link
outage (between 30 s and 32 s) during a TCP Wave connection
running, and the simulation output is shown in Fig. 20. At 30 s,
although link is off, TCP Wave continues to transmit new bursts
until RTO expires (after about 1 s). Therefore, TCP Wave starts retransmission of all the unacknowledged segments at 31 s, restoring
default parameters. In the simulated scenario, the rst two retransmitted bursts are lost again due to the outage persistence. On the
contrary, the third burst successfully arrives at the destination and
triggers a cumulative ACKs covering also some segments received
before the link outage and whose ACKs were lost. This cumulative ACK resets RTO timer, refreshes transmission parameters and
causes a jump in the burst sequence. As a result, after 1 RTT from
the link reactivation, TCP Wave transmits at the maximum rate. Of
course, all the segments proactively transmitted during outage are
retransmitted and RTO phase is concluded just after the recovery
of the last segment transmitted upon the RTO expiration.

6.6. RTT-Fairness
A very interesting network setup envisages a TCP Wave highlatency connection sharing the bottleneck link capacity with a TCP
Wave low-latency one. To this purpose the reference scenario in
Fig. 3 has been modied setting the bottleneck delay (Router1
Router2 link) to 5 ms, while only the delay on the link between
Node2 and Router1 has been increased to 240 ms. Denitively, TCP
ows coming from Node1 experience a minimum RTT of about
20 ms, and TCP ows from Node2 experience a minimum RTT of
about 490 ms.
As largely reported in literature, such a scenario drastically impairs traditional TCP performance, since ACK-clocked cwnd operations are sensitive to the experienced RTT: the higher is RTT the
slower is cwnd growth. As a consequence, bottleneck capacity is
mostly got by the low-RTT ows. Several TCP-based proposals have
solved, or at least mitigated, such a problem using some equationbased enhancements to the TCP congestion control. This is the case
of both TCP Cubic [22] and TCP Hybla [6]. TCP Cubic algorithms
positively impact on fairness as reaction of losses, while TCP Hybla

A. Abdelsalam et al. / Computer Networks 112 (2017) 122143

137

Fig. 19. RTT ination during retransmissions.

Fig. 20. RTO management in TCP Wave.

adopts an adaptation factor to the traditional TCP control formulas, using a reference RTT of some dozens of milliseconds. The aim
of the next simulation is to demonstrate that TCP Wave presents
an optimal RTT-fairness capability as well. Simulation envisages a
TCP Wave transfer from Node1 to sink1 lasting 60 s. In the middle of this transfer, Node2 accesses shared link to transfer data to
the sink 2. Figs. 21 and 22 show BURST transmission evolution and
RTT, respectively, for both TCP Wave connections. The overall behavior is similar to that observed when competing ows see the
same RTT. In the rst 30 s, the low-latency connection gets all the
available bandwidth. When also the high latency connection starts,
both immediately transmit at the same rate. Average throughput
along the whole simulation is 1.965 Mbit/s. Then, both full utilization and optimal fairness are conrmed.
7. TCP Friendliness
To evaluate TCP Wave applicability in current networks, it is
of paramount importance to evaluate its friendliness with standard
TCP, based on the Additive Increase Multiple Decrease (AIMD) con-

gestion control. In fact, in short or medium-terms scenarios, coexistence with standard TCP ows must be assumed. As a target, a
fair and stable sharing of the bottleneck capacity must be provided
at the steady state. Traditionally, a new TCP-like transport protocol
is TCP-friendly when, at the steady state, it gets an amount of
the bottleneck capacity equal or lower than that exploited by standard TCP. Herein, TCP NewReno implementation has been taken as
standard protocol reference and then considered for simulations.
To test friendliness, a simulation scenario relying on the basic
setup shown in Fig. 3 was congured. The only variations regard
the MTU size and bandwidth in access link between TCP-based
sources and Router 1. MTU has been increased up to 10 0 0 bytes
in order to improve on TCP New Reno congestion control, which
increases rate on a packet-basis. Instead, bandwidth of the access links has been increased from 2 Mbit/s to 10 Mbit/s in order
to emphasize buffering dynamics at the bottleneck link between
Router 1 and Router 2.
Tests envisage a TCP Wave and a TCP New Reno connection simultaneously running over a very long time (1 h) and competing
for the bottleneck capacity. For the sake of result clearness, bulk

138

A. Abdelsalam et al. / Computer Networks 112 (2017) 122143

Fig. 21. Burst scheduling of different-RTT connections.

Fig. 22. RTT on different latency links.

data transfers are performed over both TCP connections. Therefore, the long-term interaction between the two protocols can be
fully characterized, avoiding any limitation coming from application methods and congurations. Test outputs address a number
of parameters (and plots) useful for an exhaustive analysis of the
TCP Wave fairness as well as the overall protocol eciency:
Transmission Rate plot - Useful to show the smoothed bandwidth sharing over time of the two competing ows. This
output represents the main friendliness indicator; as much
throughput of the two ows is similar and close to half of the
available bottleneck bandwidth as more TCP Wave can be considered friendly with TCP New Reno.
Instantaneous Rate detail plot - this plot is a ner zoom of the
previous one, aimed to highlight dynamics of the interaction
between TCP Wave and TCP New Reno.
Re-transmission detail plot - A cycle of the interaction between
TCP New Reno and TCP Wave terminates with a buffer overow, which is the consequence of the reactive approach of TCP
NewReno congestion control. Basically, TCP NewReno increases

more and more its cwnd until losses are experienced. This plot
focuses on how TCP Wave manages the recovery of multiple
losses up on a buffer overow.
Cwnd/Equivalent Cwnd plot - Congestion Window (cwnd) is the
main parameter for all the standard TCP versions to regulate congestion control operations. The higher is the cwnd the
higher is the transmission rate. Then,

cwndopt [MT U] =

bandwidth[bytes/s] RT T [s]
MT U[bytes]

represents the optimum value to achieve the whole bottleneck


bandwidth. To opposite, TCP Wave leverages on BURST parameter for transmitting, so that a comparison with traditional cwnd
trend is not straightforward. However, it is possible to dene an
equivalent cwnd for TCP Wave, as the number of MTUs transmitted over the current RTT:

cwndeq [MT U ] =

BU RST [MT U ]
RT T [s]
T xT ime[s]

RTT - Round Trip Time provides a meter of the overall network conditions. When RTT is higher than its minimum value,

A. Abdelsalam et al. / Computer Networks 112 (2017) 122143

139

Fig. 23. average rate and overall throughput with = 150 ms.

Fig. 24. Instantaneous rate with = 150 ms.

the bottleneck capacity is saturated and bottleneck buffer stores


data. In general, a continuous RTT oscillation with periodical returning to values close to the minimum one represents a clear
evidence that the two ows achieved their steady-state.
Fig. 23 shows the rate trend of the two competing ows together to the overall throughput measured on the bottleneck link.
Curves have been obtained averaging rate over a window of 500
instantaneous samples in order to make overall trend more clear.
Results mainly show how the two ows share capacity almost
equally overall the simulation, oscillating around the ideal value
of 1 Mbit/s each. Therefore, TCP Wave friendliness is conrmed
as well as its capability to achieve a steady-state when competing with other ows. In addition, total throughput is always close
to 2 Mbit/s, also conrming an optimal resource utilization. This
is a very important result, since it conrmed a possible fast deployment of TCP Wave implementations running together with traditional TCP ones. Last but not least, achieved performance indi-

rectly demonstrates how TCP Wave avoids inter-protocol starvation, although competing for bandwidth with a greedy protocol
that tends to increase its rate more and more, irrespective of growing congestion. In fact, the Adjustment Rate algorithm allows TCP
Wave to periodically reset its transmission parameters contrasting
the greedy TCP NewReno standard behaviour.
To further clarify such interaction details, Fig. 24 shows a snapshot of the instantaneous throughput over the rst 50 s of simulation. TCP NewReno rate is computes as cwnd/RTT and presents
enough small oscillations due to the variation of the experienced
RTT, which is periodically inated upon the transmission of TCP
Wave bursts. To opposite, TCP Wave burst transmission leads to a
completely different trend based on succession of periods where
rate is equal to the overall bottleneck capacity, at the beginning
of the Tracking Mode, with periods where rate is taken down to a
minimum value as envisaged by the Adjustment Mode algorithm;
eventually, both transfer with a similar average rate as shown in
Fig. 23. As a conclusion, the TCP NewReno cycle is bounded be-

140

A. Abdelsalam et al. / Computer Networks 112 (2017) 122143

Fig. 25. Zoom on TCP Wave error recovery during buffer overow.

Fig. 26. Congestion Window comparison with = 150 ms.

tween two loss events and then with a time-scale proportional to


the size of the bottleneck buffer. To opposite, TCP Wave cycle is
driven by the alternation of Tracking and Adjustment algorithm
triggering, and then relies on a smaller time scale as shown by
frequent rate oscillations in Fig. 24.
Fig. 25 gives a focus on the strategy adopted by TCP Wave to
recover from multiple losses when buffer overows occur (in this
scenario this overow is due to TCP NewReno). First, it is useful to
notice that TCP Wave most likely loses an high number of packets during a buffer overow due to its transmission in bursts. To
opposite, TCP NewReno at steady state works in Congestion Avoidance and than it increases its cwnd by one every RTT: new packets transmission are spread over time since they are scheduled
upon the reception of ACKs. As a consequence, TCP NewReno in
most cases loses a single packet when overow happens, as conrmed by cwnd trend in Fig. 26 which shows that TCP New Reno
cwnd is simply halved once: this behavior indicates that fast retransmission and fast recovery algorithms are triggered to recover
from a single loss.

As already discussed in Section 6.5, TCP Wave recognizes a lost


packet when receiving at least 3 dupACKs. This allows it to recover also from multiple losses, since the continuous and proactive
transmission of bursts during loss recovery makes dupACKs generation available for consecutive losses. As retransmissions take place,
a packet per RTT can be recovered. Of course, when receiving a
new ACK related to a packet re-transmitted and successfully received, measured RTT is inated by the time needed to perform
the retransmission. This RTT sample is signicantly increasing the
computed avgRTT which, after a number of recovery, most likely
lead to exceed threshold then triggering Rate Adjusting algorithm. TCP Wave goes back in Tracking Mode only after all the
losses are recovered and measured RTT is back to a lower value.
Denitively, entering in Adjustment Mode during recovery of multiple losses allows to reduce transmission rate and consequentially
the buffer occupancy in order to recover from the congestion status which caused the overow event. Of course, the Adjustment
Mode duration is proportional to the number of packets to recover.
Since the buffer size is twice the bandwidth-delay product, just

A. Abdelsalam et al. / Computer Networks 112 (2017) 122143

141

Fig. 27. RTT evolution in case of competing ows.

before an overow, RTT is three times the physical one (around


500 ms). This is conrmed in Fig. 27 where RTT trend is shown.
RTT periodically grows up to around 1500 ms and then drop to
its minimum value. The return at the minimum RTT value after an
overow is allowed by the TCP Wave strategy of performing in Adjustment Mode for the whole loss recovery period.
Looking to the cwnd comparison shown in Fig. 26, it is clear
that although TCP Wave provides an equivalent cwnd oscillating
over a range larger than the TCP New Reno cwnd, both of them
tend to a similar average value. This evidence further conrms the
fair sharing of the available resources at the steady-state.

8. Conclusion
Future Internet must face up new challenges not approached by
traditional transport protocol design. In particular, TCP has been
designed with assumptions not valid any longer in current networks, such as low latency, no channel packet losses, no dynamic
link handover, etc. In this context, TCP Wave proposes a new approach to provide a reliable transport services over today networks.
New requirements and characteristics of current networks are addressed through an innovative burst-based transmission paradigm,
as an alternative of the traditional TCP window-based approach.
This paper presents the new protocol characteristics and methods implemented in the Network Simulator NS-3. In particular, the
most challenging network events are tested through a exible simulation setup in order to validate TCP Wave suitability to transmission scenarios not envisaged at all by traditional TCP implementations. In fact, simulation setup was designed with the aim to reproduce the most challenging dynamics faced in most innovative
networks: heterogeneous networks, leaky links, link handover, SDN
changes, multi-protocol bottleneck competition, etc. Simulation results conrmed how TCP Wave is suitable for the use in these new
networks. In particular, achievements highlighted two main protocol characteristics: i) the ability to quickly match the available
bandwidth, useful in case of short transfers as most of Internet
ows, and ii) the ability to converge to a fair and stable steadystate in case of longer ows either stand-alone or competing with
other ows. Last, but not least, TCP Wave ows resulted friendly
with standard TCP, allowing a fair bottleneck bandwidth sharing in
case of competition.

References
[1] M. Allman, V. Paxson, E. Blanton, TCP Congestion Control, Network Working
Group RFC 5681, 2009.
[2] W. Stevens, TCP/IP Illustrated, Addison Wesley, vol 1 (1994).
[3] G. Huston, TCP in a wireless world, IEEE Internet Comput. 5 (2) (2001) 8284.
[4] M. Luglio, C. Roseti, F. Zampognaro, Performance evaluation of TCP-based applications over DVB-RCS DAMA schemes, Int. J. Satell. Commun. Netw. 27 (3)
(2009) 163191.
[5] C. Barakat, E. Altman, W. Dabbous, On TCP performance in a heterogeneous
network: a survey, IEEE Commun. Mag. 38 (1) (20 0 0) 4046.
[6] C. Caini, R. Firrincieli, A new transport protocol proposal for internet via satellite: the TCP hybla, in: Proceedings of ESA ASMS conference 2003, 2003.
[7] C. Cassetti, M. Gerla, S. Mascolo, M. Sanadidi, R. Wang, TCP Westwood: end
to-end congestion control for wired/wireless networks, Wireless Netw. J. (8)
(2002) 467479.
[8] F. Belli, M. Luglio, C. Roseti, F. Zampognaro, Evaluation of TCP performance over
emulated DVB-RCS scenario with multiple RCSTs, in: Proceedings of IWSSC09
- 2009 International Workshop on Satellite and Space Communications, 2009,
pp. 424428.
[9] M. Luglio, C. Roseti, F. Zampognaro, Improving performance of TCP-based applications over DVB-RCS links, in: Proceedings of 2009 IEEE International Conference on Communications, ICC 2009, 2009.
[10] G.T. 36.331, LTE; evolved universal terrestrial radio access (E-UTRA; radio resource control (RRC); protocol specication, ETSI TS 136 331 v10.7.0 (201211)(2012).
[11] ETSI, Digital video broadcasting (DVB); second generation DVB interactive
satellite system (RCS2); part 2: lower layers for satellite standard, Draft ETSI
EN 301 545-2 V1.1.1(2011).
[12] F. Zampognaro, C. Roseti, M. Luglio, A. Detti, A. Caponi, Mobile-PEP: satellite
terminal handover preserving service continuity, The proceedings of Twelfth
International Symposium on Wireless Communication Systems, 2015.
[13] S. Salsano, N. Blefari-Melazzi, A. Detti, G. Morabito, L. Veltri, Information centric networking over SDN and openow: architectural aspects and experiments
on the OFELIA testbed, Comput. Netw. 57 (16) (2013) 32073221.
[14] C. Ververidis, P. Fragoudis, Y. Thomas, V. Siris, F.A. I. Andrikopoulos, C. Baudoin,
M. Guta, Experimenting with services over an information-centric integrated
satellite terrestrial network, IEEE Future Netw. Mobile Summit (2013).
[15] T. Lakshman, U. Madhow, The performance of TCP/IP for networks with high
bandwidth-delay products and random loss, IEEE/ACM Trans. Netw. 5 (3)
(1997) 336350.
[16] M. Luglio, C. Roseti, F. Zampognaro, F. Belli, An emulation platform for IP-based
satellite networks, in: Proceedings of 27th IET and AIAA International Communications Satellite Systems Conference, ICSSC 20 09, 20 09.
[17] Satlabs, Interoperable PEP (I-PEP) transport extensions and session framework
for satellite communications: air interface specication, Satlabs specs (2005).
[18] T. Henderson, S. Floyd, A. Gurtov, Y. Nishida, The newreno modication to
TCPs fast recovery algorithm, Internet Engineering Task Force (IETF), RFC
6582(2012).
[19] C. Jin, D. Wei, S. Low, FAST TCP: motivation, architecture, algorithms, performance, INFOCOM 2004. Twenty-third Annu. Joint Conf. IEEE Comput. Commun.
Soc. 4 (2004) 24902501.
[20] S. Floyd, Highspeed TCP for large congestion windows, Network Working
Group, RFC 3649, 2003.
[21] Y. Iyer, S. Gandham, S. Venkatesan, STCP: a generic transport layer protocol for

142

[22]

[23]

[24]

[25]

A. Abdelsalam et al. / Computer Networks 112 (2017) 122143


wireless sensor networks, in: The proceedings of 14th International Conference
on Computer Communications and Networks, 2005.
W. Bao, V. Wong, V. Leung, A model for steady state throughput of TCP CUBIC, in: IEEE Global Telecommunications Conference (GLOBECOM 2010), 2010,
pp. 16.
M. Ghobadi, S.H. Yeganeh, Y. Ganjali, Rethinking end-to-end congestion control
in software-dened networks, in: Proceedings of the 11th ACM Workshop on
Hot Topics in Networks (HotNets-XI), 2012, pp. 6166.
B. Sonkoly, F. Nemeth, L. Csikor, L. Gulyas, A. Gulyas, SDN Based testbeds for
evaluating and promoting multipath TCP, in: 2014 IEEE International Conference on Communications (ICC), 2014, pp. 30443050.
C. Roseti, M. Luglio, F. Zampognaro, Analysis and performance evaluation of
a burst-based TCP for satellite DVB RCS links, IEEE/ACM Trans. Netw. 18 (3)
(2010) 911921.

[26] M. Luglio, C. Roseti, F. Zampognaro, Transport layer optimization for cloud


computing applications via satellite: TCP noordwijk+, IEEE China Commun. 11
(12) (2014) 105119.
[27] A.A. Salam, M. Luglio, C. Roseti, F. Zampognaro, A burst-approach for transmission of TCP trac over DVB-RCS2 links, 2015 IEEE 20th International Workshop on Computer Aided Modelling and Design of Communication Links and
Networks (CAMAD), 2015.
[28] D. Constantinos, P. Ramanathan, D. Moore, What do packet dispersion techniques measure? INFOCOM 2001. Twentieth Annual Joint Conf. IEEE Comput.
Commun. Soc. 2 (2001).
[29] NS-3, NS-3 reference manual, Online NS-3 2 (2010).

A. Abdelsalam et al. / Computer Networks 112 (2017) 122143

143

Ahmed Abdelsalam received his B.Sc. degree from University of Alexandria - Egypt in 20 0 0. He received his M.Sc. in Information Security from
Royal Holloway University of London United Kingdom in 2010. Ahmed received his Ph.D. in Space Systems and Technologies from University of
Rome Tor Vergata - Italy in 2015. Ahmed Abdelsalam is now a Postdoc Researcher at the University of Rome Tor Vergata, Italy. He is involved in
research activities related to Satellite Systems, supporting some research funded projects. His research interests include network security of both
wired and wireless networks, networking protocols, and future internet.

Luglio received the Laurea degree in Electronic Engineering at University of Rome Tor Vergata. He received the Ph.D. degree in telecommunications. At present he is associate professor of telecommunication at University of Rome Tor Vergata. He works on designing satellite systems
for multimedia services both mobile and xed. He teaches Satellite Telecommunications and Telecommunications basics. He is member of the
steering committee and of the executive board of NITEL Consortium. He has several years of experience in project management, teaching and
leading many R&D activities within research institutes, universities and private companies.

Cesare Roseti graduated cum laude in 2003 in Telecommunication Engineering at University of Rome Tor Vergata. In 2003 and 2004, he was a
visiting student at Computer Science Department of University of California, Los Angeles (UCLA). From August to December 2005 he worked at the
TEC-SWS division of the European Space Agency (Noordwijk, The Netherlands). He received the Ph.D. degree in Space systems and technologies in
20 07. In 20 09, he achieved II-level master on Homeland Security at the University of Bologna. He is currently assistant professor at the University
of Rome Tor Vergata. He teaches Broadcast technologies and he is assistant professor in the Satellite Communications class, at the University
of Rome Tor Vergata. Since 2010, he teaches Principles and methods for the risk analysis at the II-level master on Homeland security (Campus
Biomedico, Rome). He is involved in national and international research projects in the telecommunication eld, and he is author of about 60
scientic publications.

Francesco Zampognaro received his Ph.D. in Space Systems and Technologies from University of Rome Tor Vergata in 2010 and his M.Sc. in
Telecommunication Engineering from University of Rome La Sapienza in 2004. He was Young Graduate Trainee for 1 year in 2006 at ESA/ESTEC
in the EUI/Telecommunications department. He is contract-professor in Space Systems at University Marconi, and was awarded a Research Grant
in Satellite Broadband multi-beam simulations, for which activities are still in progress. He is involved in research activities related to Satellite
Systems, supporting many research funded projects. His main interests are the study and simulation of Satellite Systems, in particular DVB-RCS,
covering protocols optimisation, services provision and QoS, security, resource allocation and integrated/hybrid architectures.

Você também pode gostar