Andronetra Connectivity not a Challenge

Brajender Kaur, Vinod Rajput, Dhiresh Mundada and Nilay Haryal

Computer Engineering Department Sinhgad Institute of Technology & Science, University of Pune E-mail {brajenderkaur@gmail.com, vinod.rajput214@gmail.com, dhiresh_mundada@yahoo.co.in, nilayharyal@gmail.com}
Abstract: In this paper we propose Andronetra, an application to make calling and messaging simpler for the visually challenged. The paper highlights the research and implementation details of Andronetra. The application uses various numeral processing algorithms and machine learning concepts for handwritten number-pattern recognition. Using Andronetra call and message can be made to pre-saved contacts using pattern recognition and the position of user can be retrieved using global positioning system technology. Keywords: Online processing, Offline processing, Speed Dialing, machine learning, numeral processing and handwriting recognition.

calling and messaging the user has to draw any number from 0 to 9 on his smart phone touch screen. The application then recognizes this number and asks the user if he wants to make a call or send a message. To make a Call the user has to write C on his phone screen. The application would ask the user for confirmation by asking him to press left button (Call button) else he has to press right button (End button).

I. INTRODUCTION The purpose of this research is create an application that would enable the visually challenged to use some basic features of mobile phones thus making their life a bit simpler. It would help the visually challenged to be connected with the world. The research is devoted to find an algorithm which would require less time for pattern recognition, would have a small database and yet be efficient. The two algorithms 13 point feature extraction and 23 point feature extraction are described in detail. The reason for using Android operating system based mobile phones is that Android is an upcoming open source technology. Our research is dedicated to those millions of differentially abled people who the world has wrongly labeled as disabled. We hope that through our project we would contribute something to the society and help make the lives of millions of visually challenged people easier. Through this application speed dialing that is dialing of any of the ten pre-saved phone numbers just by using a single digit number is implemented. The user can also send to these numbers any of the ten presaved message templates. To use the feature of

Fig. 1: Flow of Application. To send a message the user has to write M on his Android phone. The application would then read the message templates and then the user has to write the number of the message template which he wants to send. The application would ask the user for confirmation by asking him to press left button else he has to press right button. To retrieve the current position the user makes a P on the screen and a voice informs the user about his/her current position. Then the user can choose to message this information to his/her contacts. Audio help is provided throughout the use of application.

II.

RELATED WORK

There are many applications built in android for the visually challenged, but maximum do not address the basic problems faced by them.The applications, though really good in their approach did not cater for the basic needs of a blind person.

Maximum are based built for navigation purposes and others are built for simple mobile operations and are based on voice support.After personally trying many such applications based on voice support, such as Vlingo Virtual assistant and speaktoit we personally felt that it was hard for the application to understand the accent of many. Another project available for the visually challenged is thevOICe for Android, it is a universal translator for mapping images to sounds. Once started, the vOICe for Android continuously grabs sounds snapshots from the camera. Each camera snapshot is sounded via a polyphonic left-to-right scan through the snapshot while associating height with pitch and brightness with loudness.

the maximum X co-ordinate and the number of columns are equal to the Y Co-ordinate. Initially all the values of the array are set to one. As soon as the user leaves the screen after drawing the pattern the array is saved and further processing is carried out. Thus the output of the Binarization module is a binary value array.

All the projects discussed above do not address the connectivity problem faced by the visually challenged and thus the visually challenged are unable to do basic mobile operations such as calling and messaging. This made us come up with the idea of developing an application that would enable people to use the basic operations of mobile. Thus this application is made keeping in mind the vocally and the visually challenged.

Fig. 2: Binarization

3.2CROPPING MODULE The array obtained after binarization is an input to the cropping module. This array needs to be cropped because there are very high chances that the user while drawing the pattern has not used the entire screen but the array contains information about the entire screen.The user being visually challenged is unaware of the screen size and may either tend to draw the pattern somewhere in a corner of may tend to draw the image either too small or too large. These conditions can be handled by cropping the array. Rows and columns where no value is set to zero are cropped from the existing array, thus a new array is obtained. The reason to crop rows and columns where no value is set to zero is that these rows and columns were not used when the pattern was drawn and hence contain no useful information.

III.

DESIGN DESCRIPTION

As the user draws a pattern on the screen, the system performs online processing on the pattern. The coordinate values of the screen are extracted and binary values are stored in an array. Further cropping is done and 13 point feature extraction is performed on the array. Then the array is compared with the database to find an appropriate match. Most frequently used numbers are cached in the memory thus implementing machine learning. The design of the modules in the system is discussed below: 3.1 BINARIZATION MODULE The system uses online processing to process the pattern drawn by the user as the user draws the pattern the co-ordinate values are extracted and the respective position in the array is set to zero, The size of the array is decided dynamically by retrieving the screen resolution The number of rows are equal to

13 POINT FEATURE EXTRACTION AND FEATURE MATCHING MODULE In the feature extraction module the cropped array is divided into 13- point as shown in fig. 3. The number of rows is divided by four and the number of columns is divided by two thus we obtain the parts one to eight. Part nine is part three and four taken together and part ten is part five and six taken 3.3

together. All the parts of the first column together form part eleven and the second column forms part twelve. The entire array taken together is part thirteen.

compared with the entries in the cache. If no match is found with the deviation within set limits then the array is compared with the entries in the database. This will not only ensure faster response but greater efficiency. In general cases the hit ratio is much higher than the miss ratio. Thus using the concept of caching the learning module is implemented.

Fig. 3: 13 Point Feature Extraction We count the bit set to zero in each part of the array and store these values in a separate 1*13 array. We use this array for comparison with the database. In the database, there are pre saved 1*13 arrays for each number. The database has several entries for each number. Matrix is stored in the database. Hence the database has a variety of entries thus improving the efficiency of the system. The array which is currently obtained is compared with these existing arrays and the one which gives minimum deviation is chosen as the final result. To calculate the deviation we first find the difference of each of the thirteen points of the obtained array with the array in the database then we sum up the difference obtained from each point to get the total deviation. If the obtained array is [0 7 6 5 10 11 6 5 9 20 21 28 49] and the array in database is [0 8 4 6 10 10 6 4 10 20 20 28 48] then deviation is |0-0| + |7-8| + |6-4| + |5-6| + |10-10| + |11-10| + |6-6| + |5-4| + |9-10| + |20-20| +|21-20| + |28-28| + |49-48| = 9.Thus the deviation in this case is nine. Fig. 4: Overall Flow Of Application

IV.

23 POINT FEATURE EXTRACTION

After doing research on 13 point feature extraction it became clear that maximum density of the pattern is in the center. This fact helped us to try to improve 13 point feature extraction and we propose a new algorithm called 23 point feature extraction. In this the given array is divided into 23 parts as shown below. The center region appears in multiple parts as the density of the pattern is more there. We count the bit set to zero in each partof the array and store these values in a separate 1*23 array. This array we use for comparison with the database. In the database, there are pre saved 1*23 arrays for each number. The database has several entries for each number. The array which is currently obtained is compared with these existing arrays and the one which gives minimum deviation is chosen as the final result. To calculate the deviation we first find the difference of each of the thirteen points of the obtained array with the array in the database then we sum up the difference obtained from each point to get the total deviation.

3.4 LEARNING MODULE There is also a module which keeps an account of the commonly used numbers by the user and based on which it updates itself every time the user makes a pattern. This module makes use of the concept of caching that is the numbers which are most frequently used their arrays are stored in the cache. When an array needs to be compared it is first

Fig. 5: 23 Point Feature Extraction

V.

MAJOR FINDINGS AND IMPLICATION

Though 13 point feature has slightly lower efficiency its time and space requirements are much lower hence it has been implemented in the system.

For pattern processing we have done research on two major processing approaches namely online processing and offline processing.

Parameter
In online processing the processing of the image begins as soon as the user starts drawing the image on the screen. The co-ordinates are captured runtime using touch events. There is no need to wait till the entire image has been drawn. Binarization starts as soon as the user starts making the pattern. In offline processing the processing on the image begins after the entire image has been drawn. It uses the canvas feature. There is need to wait till the entire image has been drawn. Once the image has been drawn the snapshot of the screen is taken. This is then stored in either the phones main memory or the external storage card. Only after this step can processing begin. Considering the time and space requirements of the application it is better to use online processing as it is faster than offline processing and there is no need to save the pattern as an image. The two feature extraction algorithms namely 13 point feature extraction and 23 point feature extraction were evaluated based on a few major parameters and the finding of the research is summarized in the table shown in Fig 6. Space required

13 Point
To store a 1*13 array space required is less Comparison is cheaper as the values to be compared are less As the array is divided into 13 parts its processing is faster Density of the features extracted is low The algorithm has slightly low efficiency

23 Point
To store a 1*23 array space required is more Comparison is expensive as the values to be compared are more As the array is divided into 23 parts its processing is slower Density of the features extracted is high The algorithm has high efficiency

Time for Comparison

Time for Processing

Features Extracted

Efficiency

Fig. 6: Comparison of 13 Point and 23 Point Feature Extraction

VI.

CONSTRAINTS

X.

REFERENCES

There are a certain constraints of this application that are listed below: 1. Using this application call can be made to only ten pre-saved numbers Reason: If more than ten speed dial numbers are allowed the user will have to remember all the respective numbers which is not feasible. Setting needs to be done by a person with normal vision. Reason: As the visually challenged would not be able to see the contents on the screen changing the setting by him/her is not possible. The user would be required to know the pattern of numbers used in English language. Reason: The application can only recognize numbers of the English language.

[1] Hyun Kang and Hang Joon Kim , Design Of An Interface onPDA for Korean Language ,IEEE Transactions on Consumer Electronics,Vol. 46, No. 3, AUGUST 2000. [2] JinhaiCai and Zhi-Qiang Liu,,Integration of Structural and Information for Unconstrained Handwritten Numeral Recognition., IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 21, No. 3, MARCH 1999. [3] PyeoungKee Kim, Improving Handwritten Numeral RecognitionUsing Fuzzy Logic. [4] K.B.M.R. Batuwita, G.E.M.D.C. Bandara Handwritten Numeric Characters [5] http://mathworld.wolfram.com/P Statistical Recognition., Fuzzy Recognition of Offline P-Problem.ht [6]http://www.seeingwithsound.com/android.htm

2.

3.

VII.

FUTURE SCOPE

The future enhancements in the application include 1. Sending messages to the pre-saved speed dials using voicemails. 2. Providing the user the feature to operate the other functionalities of phones, like camera, video recording etc. VIII. CONCLUSION

This application through numeral pattern matching, handwriting recognition and voice messaging would hence make speed dialing and messaging from smart phones possible and simpler

IX.

ACKNOWLEDGEMENT

We would like to offer our sincere thanks to our guide Prof.(Ms) Geeta Navale, Head Of Computer Department, Sinhgad Institute Of Technology and Science.We would also like to thank Persistent Systems for their guidance and sponsorship.