CLICKER – AN IPTV REMOTE CONTROL IN YOUR CELL PHONE
Rittwik Jana, Yih-Farn Chen, David C Gibbon, Yennun Huang, Serban Jora, John Murray, Bin Wei
AT&T Labs Research, 180 Park Avenue, Florham Park, NJ, 07932, USA.
ABSTRACT
This paper investigates a novel concept of providing seamless
control and portability of an IPTV viewing session. A solution
employing a middleware system, a secure hardware token and a
cell phone are used to demonstrate how an IPTV session can be
securely controlled remotely and moved between multiple viewing
stations. We build a prototype of the system and demonstrate its
flexible features. Depending on the user’s protocol of choice, most
remote control operations from a mobile device took less than 5
seconds to execute. An interesting capability of previewing content
of other channels via the user’s device while still continuing to
watch the program on the viewing station show a difference from
today’s IPTV offers. Finally for mobile content delivery, we
address a problem of dynamic device profile selection and content
adaptation using a classification algorithm to match the best
content alternative destined for a mobile device.
1. INTRODUCTION
IPTV describes a system where a digital television service is
delivered using the Internet Protocol over a network infrastructure,
which may include delivery by a broadband connection. For
residential users, IPTV is often provided in conjunction with
Video on Demand and may be bundled with Internet services such
as Web access and Voice over IP (VoIP). The commercial
bundling of IPTV, VoIP and Internet access is referred to as a
Triple Play. Adding the mobile voice service leads to the
Quadruple Play moniker. IPTV is typically supplied by a
broadband operator using a closed network infrastructure. This
closed network approach is in competition with the delivery of TV
content over the public Internet; however, the latter is often lacking
a guaranteed quality of service [1]. Internet TV and DVR
capabilities have become very popular in the past few years [2][3].
Mobile phones are being proposed to control the smart home and
to offer a personalized service [6][8].
In this paper, we are interested in providing to the mobile user the
capability of controlling an IPTV session from his mobile device, a
concept that we define as “out-of-band” remote control – “clicker”.
The solution relies on a mobile device acting as a remote control, a
secure token [4] to authenticate user and move IPTV sessions, and
a middleware server acting as an intermediary or proxy between
the user and the IPTV server. Mobile content delivery is an
important issue that needs to be discussed in association with this
application. Information (multimedia clips etc.) sent to the device
needs to be adapted and matched to a particular device profile. We
propose a mechanism on how content is adapted using a
middleware system that best matches the user’s device.
2. MOTIVATION
Why do we need “seamless control and portability” of your IPTV
viewing experience? The answer to this question relies on the
promise of quadruple play. Traditionally, users equate
entertainment to simply watching or viewing a program on their
television set. IPTV is more than just “viewing” programs. It aims
to provide a live interactive session with an enriched entertainment
experience. Features such as video on demand (VOD), digital
video record (DVR), instant channel change (ICC) and multiple
pictures in picture (PIP) are some of the capabilities of a well
designed IPTV solution. We propose to use the combination of a
secure token and a mobile device that can perform the associated
“control” and “move” operations.
Typically in IPTV systems, the set-top box contains a unique
hardware identifier which is registered with the service provider as
part of the provisioning process in order to provide a basic
customer identity capability. For traditional cable systems, a
CableCard can fulfill a similar purpose and support interoperability
so that the user can choose among terminal equipment vendors as
mandated by the telecommunication act of 1996 [7]. The two main
problems with box-level identification are the lack of mobility and
insufficient specificity for personalization down to a particular
family member. We propose to address these shortcomings by
moving the identity and authentication management beyond the
set-top box to a truly personalized device, the ubiquitous mobile
phone.
Seamless control: To enjoy most of these capabilities, currently the
user typically employs a remote control that is associated with the
immediate viewing station (in-band). A remote control is not
universal and that it cannot be associated with any screen on which
the user would like to watch his IPTV program. A typical use case
would be that a mobile user navigates the electronic program guide
(EPG) and schedules a network recording of a particular show. The
user then instructs the IPTV server to send him a one-minute
summary or “mobisode;” mobisode is an episodic programming
produced specifically for the user’s mobile device.
Seamless portability: For example, in a three screen strategy (set-
top box, PC and PDA), where the same content is repurposed and
sent to any screen, how will a user on-the-go perform a switch and
thereby handoff a live TV session on a set-top box enabled TV to a
WiFi enabled PC or a 3G enable PDA? Another use case occurs
when an IPTV service subscriber visits his friend’s house (non
IPTV subscriber) and would like to enjoy a TV program on his
broadband enabled PC.
To this end, for seamless control we design an “out-of-band” remote control panel. User viewing profile (e.g. last visited
capability using a cell phone that speaks or hosts multiple broadcast TV channel or the time of a VOD session that was
protocols (HTTP, VOICE, EMAIL, SMS, IM). We couple this paused before handoff) can be accessed from the eToken to
capability with a secure hardware token to move and restore a initialize the media session.
session across viewing stations. The system architecture is Remote Control: In addition to the software remote control, the
discussed in section 3, followed by a prototype development in user can perform similar remote control capabilities (e.g. channel
section 4. Section 5 discusses content adaptation and transcoding change, browse EPG info etc.) from his personal mobile device.
solutions followed by conclusions and future work in section 6. Voice: Upon launch of the applet, the IPTV server will instruct the
middleware server (MxM) to make an outbound phone call to the
3. SYSTEM ARCHITECTURE user’s cellular phone. The cellular phone is intended to be used as
the remote control device. An optional PIN can be used to provide
This section briefly outlines the architecture used to verify the additional authentication if required. Commands can be issued via
aforementioned concepts. There are three interfaces in the the cell phone keypad using DTMF or natural speech. An
architecture, namely authentication, media flow and interactive voice response (IVR) engine can interpret the DTMF
control/signaling as shown in Figure 1. The interfaces exist tones or speech utterances to trigger navigational commands seen
between a) the user secure token and the IPTV application, b) the on the applet. A voice session can be initiated using either a circuit
IPTV server and the middleware server and c) the user’s (out-of- switched (PSTN) call or a packet switched VOIP call.
band) remote control and the middleware server. Other protocols: Alternatively, remote control operations can also
be issued by means of a data mode (e.g. mobile WAP
We design the architecture with the following goals in mind: 1) microbrowser, instant messaging (IM), or short messaging (SMS)).
Network-based solution, 2) Technology Agnostic, 3) Open In the case of WAP/HTTP, the mobile user visits a web site that
interfaces, 4) Intelligent processing of multimedia data to match presents the remote controls, or communicates to the IPTV clicker
users’ device profiles and 5) Protect data and end user privacy instant messaging service buddy that translates typed commands to
using two factor authentications. To illustrate the key entities and the IPTV server or sends SMS messages. Various protocols are
interfaces in the architecture, we consider the following steps. supported by the middleware server and serve as an intermediary
between the user and the IPTV server.
IPTV server Auth
Media 3.1. Middleware platform, MxM
Control
Content
The middleware platform, (see Figure 2) is a system that contains
3 gateways, servers, a message switch and databases. Gateways send
Super Head End
Auth
and receive messages to and from devices using different protocols
2
Middleware
Media Server
(e.g., http, mail, sms, mms, voice, fax, SIP, instant messaging,
Lightspeed
/ etc.). Requests received at these gateways are authenticated to
Internet
PSTN identify the sender, the user agent, and device profile, and then
/ transmitted through the message switch to any of the backend
Internet
MxM servers. Each server hosts an identical set of “infolets” that
1 Remote Control via Cellphone implement specific application logic (e.g. clicker remote control
eToken
TV-settop laptop PC Mobile
infolet) and provide access to one or more external services (e.g.,
Web IPTV servers). An infolet’s output needs to conform to the
IPTV Viewing stations
destination delivery context for a session established for the user’s
Voice
call IM device.

VOIP Gateways
Viewers with cellphones call SMS

Infolets
Figure 1 - Logical architecture http

Servers IPTV service

Authentication: Upon insertion of the secure token (eToken), the mail Clicker Infolet
user enters their single eToken password or PIN. The user gets Location service
secure, reliable, two-factor authentication commonly identified by
“something you have” in addition to “something you know,” which mms Blogs
Message
is regarded as more superior to a single factor password-based Switch
authentication mechanism. If two factor authentication is not Publication Infolet
RSS sources

required (e.g. informational public accessible video), the token can voice
be inserted and authenticated against the IPTV platform without Aggregator Infolet

PIN entry.
SIP
Content Blog Profile
Database Database Database
Media Flow: Upon successful authentication, a browser would be
content server
launched on the viewing station and redirected to the IPTV server
URL. An applet would then be repurposed according to the user’s Figure 2 - MxM middleware platform with Clicker infolet
profile and displayed on the browser together with a software
The MxM platform offers support for information transcoding
(format conversion) in the form of a framework that can be used by
the infolet provider. Further detail on MxM can be found at [5]. A
specific infolet named “clicker” was developed to relay and
translate user commands to the IPTV server.
4. PROTOYPE AND RESULTS
Both the WAP and Voice gateways are “forward-only” elements in
the MxM architecture and are used to initiate user-to-MxM dialog.
Subsequent dialogs are directed through the MxM HTTP gateway
with the correct delivery context. Step 4 contains the interactions
between MxM and the delivery network interface elements: for
WAP, a WAP’s push proxy gateway component; for Voice, an
interactive text-to-speech telephony platform.

This section discusses a prototype implementation of the concepts Viewing Station

User device MxM IPTV
Set-top box + eToken server
introduced. Figure 3 shows the prototype design for controlling an WAP/Voice Clicker
IPTV session. The implementation was based on Java with live gateway infolet
content being streamed out of a media streaming server on HTTP
particular multicast groups, synchronized and scheduled using gateway
0
XMLTV EPG feeds. Users have the ability to navigate between 1
channels, lookup information from the EPG and push preview 2

clips to cell phones as MMS messages. 4

3

6
5 7
Content Delivery
Channel
Channel 1 Multicaster 8
9
multicast 10

Ch1 udp@ Channel 2 Schedule content

1’
multicast 2’
3’
udp@ Content Delivery
Ch2 IPTV server
tune udp@xx 4’
IPTV 5’
6’
Client
login/logout/tune
Figure 4 – Call flow sequence diagram
tune,schedule
Ch1
5, 6, 7: The user password is submitted by the user through MxM
to the IPTV authentication component which will match it with the
HTTP Clicker
tune Ch1
gate infolet information received in 1 and its stored authentication information.

Voice 8, 9, 10: In case of positive authentication, after content delivery is

gate
initiated, associated channel information and control page are
SMS presented to the user. Otherwise, an authentication error page is
gate Middleware sent back to the user device.
server
1’, 2’, 3’: Remote Control commands are received from the user
(e.g. channel change, fast forward etc.) and are intercepted and
Figure 3 - Prototype design schematic interpreted by MxM on route to the IPTV server who adjust the
content delivery channel accordingly 4’, 5’, 6’: New channel info
4.1. Call Flow Scenario and requested content is presented to the user.
Figure 4 depicts a typical call flow scenario utilizing the
aforementioned interfaces that pertains to controlling an IPTV
session together with a secure token.

Bootstrapping
0: Service callback registration. MxM's Clicker infolet service
registers itself with the IPTV Media Server.

Authentication support
1: One time generated pin is sent from eToken (plugged into
multimedia terminal) to the IPTV authentication service (this
message also contains user identification information)

2: IPTV server calls back MxM requesting authentication

completion for the user indicated in the step 1.
Figure 5 – Prototype screen shots
3, 4: MxM Clicker infolet pushes a request to the user through the
MXM WAP/Voice Gateway containing an authentication page.
Figures 5 shows EPG navigation (e.g. previous channel, next
channel) and show selection from a WAP and HTTP mode based
remote control user interface.
Observations: With the prototype system, we tested the usability of
the clicker concept. Remote control operations via a cellular phone
varied between 1 to 5 seconds. In particular, control operations
like channel change took longer to execute than viewing EPG
information. A channel change operation initiated by the cellular
phone traverses the middleware server which notifies the IPTV
server and subsequently IPTV client whereas the EPG information
is fetched directly from the IPTV server. An interesting capability
of previewing content on other channels via the user’s device while
still continuing to watch the program on the viewing station show a
difference from today’s IPTV offers. The out of band channel can
also be used for interactive sessions with the IPTV server that
requires just-in-time user involvement (e.g. voting, shopping etc.)
5. DEVICE PROFILING AND CONTENT
ADAPTATION
Imagine while browsing the EPG on one’s mobile device, the user
would like to preview a certain selection by means of a video clip
sent to him as an MMS message. The Clicker application performs
content processing by utilizing the transcoding support available in
the MxM platform. This section highlights some of the technical
aspects of performing content adaptation by MxM on route to the
user’s remote control device. As part of the response to a user’s
preview request, a video clip has to be disseminated by the IPTV
server through MxM. A transcoding component either within
MxM or external to it is capable of extracting content parts
namely, the text caption, soundtrack and images. It is also
responsible for creating mobile renditions of the video and sound
content. We define the following terms:
Endpoint: The combination of the user, the device and the user
agent.
Profile: The pre-provisioned (static) attributes describing a user’s
preferences and its device and user-agent capabilities.
Delivery Context: The combination of all the attributes describing
the current interaction with an endpoint. It can be described as the
union of dynamically acquired information and static
user/device/user agent profiles.
TranscodingResource: A content part that is delivered to the user’s
device. The preview video, its text caption or the associated
soundtrack are all examples of transcoding resources.
Problem: The problem is to deliver the best matching content parts
to the user’s remote control subject to the constraints imposed by
the DeliveryContext. First, static profiles do not reflect some of the
dynamic aspects of the delivery context: network bandwidth,
current client configurations. Second, device profiles vary
considerably from user to user and transcoding the content in order
to optimally match each profile is very expensive. Given these
constraints we provide the following solutions.
Solutions: We have approached the problem by reducing it to a
multi-dimensional classification/clustering problem. The attributes
used to describe the available content alternatives and the endpoint
delivery contexts are viewed as the ‘dimensions’ of the problem.
Solution 1: Create a predefined set of alternatives for the content to
be delivered and consider them as cluster centers. Taking into
consideration the target delivery contexts, we perform a matching
operation to the selected clusters. This one-step classification
process should be interpreted in a ‘fuzzy way’: the best fitting
content alternative for a given delivery context is the closest one;
other alternatives may be usable (however, with reduced quality).
Solution 2: Perform clustering among the points defined by each
endpoint’s delivery context and limit the number of clusters to a
predefined level. This, however, requires the multimedia
transcoding unit to produce content alternatives matching the
cluster centers.
Observations: The first solution is faster as the two steps can be
executed in parallel given that the alternatives are pre-defined.
While the second solution produces more accurate results, it is
slower. The two steps namely, clustering and transcoding, must
also be performed in sequence. Increasing the level of
personalization or adaptation increases the required amount of
content processing thereby affecting performance. MxM provides a
framework supporting the first solution.
6. CONCLUSIONS
A novel concept of IPTV session portability and remote control is
investigated. The solution relies on a mobile device acting as a
remote control, a secure token to move IPTV sessions and a
middleware server acting as an intermediary between the user and
the IPTV server. A prototype is built to demonstrate remote control
capability by supporting multiple protocols (http, voice etc.). An
out of band remote control provides attractive possibilities that are
not available in today’s IPTV offers for example, fetching
additional information about the current program on the viewing
station or browsing IPTV content available on other channels.
Finally, mobile content adaptation is necessary to provide the user
with a personalized experience especially when delivering rich
media.
7. REFERENCES
[1] IPTV Wikipedia - http://en.wikipedia.org/wiki/IPTV
[2] Microsoft Inc, Microsoft TV Foundation,
http://www.microsoft.com/tv/FoundationEdition.mspx
[3] Sling Media Inc. – http://www.slingmedia.com
[4] Aladdin Inc. – http://www.aladdin.com/eToken
[5] Chen, Y-F., et al. “iMobile EE - An Enterprise Mobile Service
Platform,” Wireless Networks 9(4): 283-297. 2003.
[6] Digital Living Network Alliance (DLNA), “Digital Living
Network Alliance Home Networked Device Interoperability
Guidelines Expanded: March 2006”, March 2006.
[7] Telecommunications Act of 1996 -
http://www.fcc.gov/telecom.html
[8] Mizuno, S. “A mobile phone based authentication service for
home appliances”, demonstration at IEEE consumer
communications and networking conference, Las Vegas, Jan 12th
2007.