580.

471 PRINCIPLE OF DESIGN OF BIOMEDICAL INSTRUMENTATION CHALLENGE PROJECT REPORT Report Prepared By: Daniel Cho & Elizabeth Keene --------------------------------------------------------------------------------------------------------------------INDOOR SMART CANE FOR THE BLIND PROJECT OBJECTIVE The main goal of our project was to create a cane with sensors to provide feedback that would assist a blind person with navigating an indoor environment. INTRODUCTION To successfully navigate in an indoor environment, a blind person must be able to detect low and mid-height obstacles, as well as stairs. The most common tool that blind people currently use to navigate is the standard white cane. Therefore, we decided to modify and enhance a walking cane, since blind people already are comfortable with it.1 The standard white cane is only able to detect objects by touch. The blind user sweeps the cane back and forth in front of them. If the cane runs into an object or falls off of the edge of a stair, the blind person becomes aware of the obstacle too late. We wanted to provide additional feedback to the blind person to warn them of objects before they run into them. We accomplished this goal by adding infrared and sonar sensors to the cane that provided information about the environment to the blind person through tactile- auditory feedback. DESIGN PROCESS Part 1: Sensor Selection and Calibration One of the primary goals of our project was to provide earlier warning of obstacles to the blind person. Since the cane already reaches approximately 3 to 4 feet in front of the user, we needed a sensor with a fairly long range to provide earlier feedback than the cane tip could provide. Therefore, we chose the SRF05 ultrasonic range finder sensor, which was able to detect objects up to 3 meters away. The ultrasonic range finder sensor (sonar sensor) works by sending out a ping and then capturing the return wave if the ping hits an object. The time between the ping and the returning wave is proportional to the distance to the object. The advantage we had with the SRF05 sonar was that it came calibrated. The calibration data given by the SRF05 datasheet was used to program the Basic Stamp such that the output of the sensor was converted to the distance between the sensor and the object that it senses.
1

After conversations with experienced engineers and users of blind canes, we gathered enough information to conclude that the standard cane should be used as our base model. Other ideas were recommended (such as sensorembedded gloves) but the practical and aesthetic value of such designs was minimal. Blind users expressed the need to not feel too alienated by an awkward design that would only call unnecessary attention. Also, and most importantly, improving upon the ordinary cane deemed most favorable since it would minimize the training time required for blind users. Canes are readily available to them and most users are accustomed to the tactile experience of using a cane. A new design would require a learning process which could not be favored by users.

To detect close low objects and stairs, we determined that infrared sensors were ideal because of their close-range accuracy and ease of implementation. One IR sensor would be placed pointing forward to detect objects directly ahead, and the other sensor would point towards the ground to detect stairs. The 2 Sharp GP2D12 IR sensors were calibrated much the same as the sonar sensor; an object was placed in front of the sensor and the distance between the object and the sensor was correlated with the output from the sensor to create a calibration curve. The calibration curve was then used to program the Stamp to enable it to translate the IR sensor output to the distance to the object.

CM 10 15 20 25 30 40 45 50 55 60 65 70 75 2

Volts 2.51 1.79 1.39 1.14 0.97 0.85 0.76 0.67 0.62 0.57 0.53 0.50 0.48

80 100

0.46 0.00

Special Feature: The sonar sensor is mounted on a servo motor that scans approximately 45 degrees to the left and right of the user. This allows the user to sense objects to his or her sides without the need to sweep the cane back and forth. Part 2: Feedback Mechanisms After obstacles are detected by the sensors, their location and distance must be communicated to the user. Since the sonar sensor detects objects at further distances, it provides more frequent feedback to the user. Therefore, we determined that silent feedback, detectable only to the user, would be ideal. A simple and silent feedback mechanism that can easily be detected is vibration. Since the sonar sensor faces different directions at different times due to the rotation of the servo motor, more than one vibration motor was needed to communicate not only that an object was within a certain distance of the user, but also the location with respect to the user of that obstacle. Three vibration motors were mounted on the handle, one for left, center, and right of the user respectively. The center vibration motor was positioned beneath the thumb of the user, since it provides the most relevant and important feedback. The left vibration motor was mounted beneath the index finger, and the right vibration motor was mounted beneath the pinky finger. With the use of the sonar sensor and feedback, many obstacles can be avoided before the user comes within close proximity of an obstacle. However, low objects below the sight of the sonar sensor must be detected with the IR sensor. This is only meant to provide the user with feedback a short time before they will contact an obstacle. It should therefore occur less frequently, but also will require more immediate action by the user. Consequently, we determined that a buzzer would provide an excellent alert. Since there are 2 IR sensors, each sensor sends feedback through a different pitch from the buzzer. Part 3: Programming the Basic Stamp 2 The next step in the project was to program the Basic Stamp 2(Stamp)2 to take readings from the sensors and provide useful feedback to the user through the vibration motors and buzzer. The sonar sensor provided digital output and could thus be directly connected to a pin on the stamp. In contrast, the IR sensors provided analog output and thus needed to be connected to an A/D converter (ADC 0831) which then connected to the Stamp. As mentioned in the sensor section, the Stamp was programmed to take the output from the sensors and use the calibration curves to convert the output to a distance. The distance output from the sensor then was compared to a reference value determined to be useful for the user through an if loop. For the sonar sensor, if an object was within a certain distance (varies by code, 1 meter for the attached) of the user at the center servo position, the Stamp was programmed to turn the pin to the corresponding vibration motor to high, thus causing it to vibrate. The left and right positions of the sonar

The Basic Stamp 2 was used for ease of programming (PBASIC) and robust computing capabilities. The stamp required less space than a PIC microcontroller and was very easy to program.

sensor and their vibration motors were programmed analogously, only with a closer object detection distance of 50 cm. (Again, varies by code and circumstances) The IR sensors also were programmed to use if loops to compare the converted distance value from the sensors to a reference value. The IR sensor facing forward, used to detect close objects at a low level, was programmed to output high to the buzzer if the detected distance was less than 30 cm. For stairs-detection, it was programmed to output high to the buzzer if the distance was greater than the typical distance to the ground, plus a buffer distance to prevent false alerts. This was designed to alert the user of a dip in the terrain, which in indoors means stairs. The greatest programming challenge was the servo motor. The speed and direction that the servo rotates is proportional to the pulse width. With some online research and experimentation, the correct pulse-widths and for loop lengths were determined (to keep the servo rotating, pulses must be sent periodically, thus necessitating the use of a for loop). Subroutines were programmed to cause the servo to rotate quickly 45 degrees to the left, center, and 45 degrees to the right. Finally, all of the sensors were integrated into a main program. The program was designed to rotate the servo between center, left, center, and right positions repeatedly. At each position of the servo, the Stamp was programmed to take a reading from the sonar sensor and provide the servo-location specific vibration feedback if an obstacle was detected. Also, it was programmed to sequentially take a reading from each of the IR sensors and provide audio feedback if an obstacle was detected. The servo then was programmed to move to the next direction and repeat the sensor measurements. Block Diagram of the Complete Smart-Cane Dynamics
SRF05 Ultrasonic Used to send pings that bounce back to the sensor when an object is detected. The time of travel is relative to the distance to an object, computed by the code. Buzzer Emits two different frequencies (stairs, center) 3 X Vibrators Indicates the position of obstacle (L, R, Center)

Basic Stamp 2 Takes the sensor input and outputs signals to the feedback

2x

GP2D12 IR Sensor Used to send pings that bounce back to the sensor when an object is detected. The time of travel is relative to the distance to an object, computed by the code.

ADC 0831 Takes the sensor input (analog) and converts it to digital output (0~255).

Encased in a plastic module powered by a 9 volt battery with voltages regulated by a 5.0 V Regulator.

Part 4: Constructing the Cane The final step was to construct the cane. First, we took the final circuit from the breadboard and soldered it to a circuit board. It was mounted, along with two 9V batteries, in a box near the top of the cane. Holes were drilled or cut into the box to allow the wires for the sensors and feedback mechanisms out. The cane itself was made of PVC piping, thus also making it simple to screw the box to the cane. Holes also were drilled into the cane to allow the wires to pass through the inside of the cane and then out of the cane at the appropriate location (the handle for the vibration motors and buzzer, and the bottom of the cane for the IR sensors). The sonar sensor was mounted on top of the servo motor using a metal bracket. The servo motor and IR sensors were mounted to the cane using custom metal C-brackets. The top of the cane was fitted with pipe insulation to provide a comfortable grip for the handle as well as a location to implant the vibration motors. PROBLEMS ENCOUNTERED 1. The main problem that stalled us for a significant amount of time was the servo motor. It drew too much current for anything other than a brand-new 9V battery when connected to the rest of the circuit. This was causing the Stamp to brown-out, and consequently our code continued to reset from the beginning over and over. This was apparent because the servo only went to the center position and occasionally to the left, the next step in the code. To correct this problem, we used two 9V batteries in parallel to power our circuit to input more current. 2. A second problem was that the vibration of a single vibration motor caused the entire cane to vibrate, making it difficult for the user to detect which side an obstacle was on. To correct this problem, we mounted the vibration motors on top of the pipe insulation so that the vibration was only felt in the local area of the motor where the users finger was positioned. POSSIBLE DESIGN/DEVICE IMPROVEMENTS 1. Practice more with the cane to determine the most useful distances to provide feedback for the user, and use this information to improve the Stamp programming. 2. Mount the vibration motors more carefully and design a more robust way to connect them to the circuit board. They are currently too fragile and have had to be re-soldered several times. 3. Also, mount the vibration motors in such a way that they can provide useful feedback to any user. Currently, they can only comfortably be used by the person whose fingers they were placed under by design. Also, they need to be positioned comfortably for use by a right or a left-handed person. 4. Add another sonar sensor in the middle of the cane to detect objects of a height between the current sonar sensor and the IR sensors. This may help the user to detect objects like chairs and tables, which were hard to detect because the legs were not a large solid object.

POTENTIAL BIOMEDICAL APPLICATIONS The potential biomedical applications of such a device could include assisting a blind person in navigating an indoor environment. A smart cane may be especially useful for the newly blind, people who were not born blind, but by some disease or accident have now become blind. Since they are used to being able to see and are not comfortable with the standard white cane, additional feedback may help them to grow more confident and mobile more quickly.

ATTACHMENTS: 1. Circuit Diagram for the entire device 2. Program written in BASIC language to program the device.

CODE ' {$STAMP BS2} basic stamp codes ' {$PBASIC 2.5} Servo CON 13 servo motor VIB1 CON 3 the following 3 are vibrators VIB2 CON 4 VIB3 CON 5 counter VAR Word counter for sonar range wDist VAR Word same for sonar range INIT CON 0 down convfac CON 29 ECHO CON 1 pulseCount VAR Word Adc0831 PIN 8 ' ADC0831 Chip Select (ADC0831.1) our ADC converters AdcClock PIN 6 ' ADC0831 Clock (ADC0831.7) AdcData PIN 2 ' ADC0831 Data (ADC0831.6) Adc0832 PIN 9 AdcClock2 PIN 10 AdcData2 PIN 11 span CON 5 ' 5 cm Per Data Point span2 CON 5 ' 5 cm Per Data Point ' -----[ Variables ]------------------------------------------------------result VAR Byte ' ADC8031 Result used for ADC conversion and IR volts VAR Word ' Volts (0.01 Increments) cm VAR Byte ' centimeters index VAR Nib test1 VAR Byte ' Values For test2 VAR Byte ' Interpolation slope VAR Word ' mV/cm between test points result2 VAR Byte ' ADC8031 Result volts2 VAR Word ' Volts (0.01 Increments) cm2 VAR Byte ' centimeters index2 VAR Nib test1_2 VAR Byte ' Values For test2_2 VAR Byte ' Interpolation slope2 VAR Word ' mV/cm between test points ' -----[ EEPROM Data ]----------------------------------------------------Vout DATA 245, 179, 139, 114, 97 calibration data for IR sensor DATA 85, 76, 67, 62, 57 8

DATA 53, 50, 48, 46, 43 DATA 0 Vout2 DATA 245, 179, 139, 114, 97 DATA 85, 76, 67, 62, 57 DATA 53, 50, 48, 46, 43 DATA 0 ' -----[ Initialization ]-------------------------------------------------HIGH Adc0831 ' Disable ADC0831 turning on ADC HIGH Adc0832 ' Disable second ADC 0831 ' -----[ MAIN ROUTINE ]----------------------------------------------------top: GOSUB Forward_Pulse look forward (to servo) PAUSE 5 GOSUB Read_GP2D12 ' Read Sensor Value GOSUB Calculate_Distance ' Convert Value To cm DEBUG HOME, "Small Distance = ", DEC cm, " cm " ,CR PAUSE 5 GOSUB Read_2Y0A02 GOSUB Calculate_Distance_2 DEBUG HOME, "Big Distance = ", DEC cm2, "cm" , CR PAUSE 5 GOSUB sr_sonar1 PAUSE 1 GOSUB Turn_Left Look to the left (to servo) PAUSE 1 GOSUB Read_GP2D12 ' Read Sensor Value GOSUB Calculate_Distance ' Convert Value To cm DEBUG HOME, "Small Distance = ", DEC cm, " cm " , CR PAUSE 5 GOSUB Read_2Y0A02 GOSUB Calculate_Distance_2 DEBUG HOME, "Big Distance = ", DEC cm2, "cm" ,CR PAUSE 5 GOSUB sr_sonar2 PAUSE 1 GOSUB Forward_Pulse (look forward again) PAUSE 1 GOSUB Read_GP2D12 ' Read Sensor Value GOSUB Calculate_Distance ' Convert Value To cm DEBUG HOME, "Small Distance = ", DEC cm, " cm " ,CR PAUSE 5 GOSUB Read_2Y0A02 9

GOSUB Calculate_Distance_2 DEBUG HOME, "Big Distance = ", DEC cm2, "cm" ,CR PAUSE 5 GOSUB sr_sonar1 PAUSE 1 GOSUB Turn_Right Look to your right PAUSE 1 GOSUB sr_sonar3 PAUSE 1 GOSUB Read_GP2D12 ' Read Sensor Value GOSUB Calculate_Distance ' Convert Value To cm DEBUG HOME, "Distance = ", DEC cm, " cm " ,CR PAUSE 5 GOSUB Read_2Y0A02 GOSUB Calculate_Distance_2 DEBUG HOME, "Big Distance = ", DEC cm2, "cm" ,CR PAUSE 5 GOTO top ' -----[ Subroutines ]----------------------------------------------------' ************************************************************************* ' The following are subroutines for the servo, IR, and sensors ' ************************************************************************* Forward_Pulse: FOR pulseCount = 0 TO 50 PULSOUT Servo, 750 PAUSE 10 NEXT RETURN ' Send A Single Forward Pulse ' Number Of Pulses To Turn ' Ping Servo Forward Pulse Value ' Refresh Delay

' ************************************************************************* ' ************************************************************************* Turn_Left: ' Left Turn, About 45 Degrees FOR pulseCount = 0 TO 20 ' Number Of Pulses To Turn PULSOUT Servo, 1050 ' Ping Servo Forward Pulse Value PAUSE 20 ' Refresh Delay NEXT RETURN ' ************************************************************************* ' ************************************************************************* Turn_Right: ' Right Turn, About 45 Degrees FOR pulseCount = 0 TO 20 ' Number Of Pulses To Turn 10

PULSOUT Servo, 450 PAUSE 20 NEXT RETURN

' Ping Servo Forward Pulse Value ' Refresh Delay

sr_sonar1: for the sonar we have 3 different routines for where the servo is PULSOUT INIT, 5 PULSIN ECHO,1,wDist wDist = wDist/convfac PAUSE 10 DEBUG DEC wDist, CR PAUSE 20 IF wDist<10 THEN if something is less than 10 cm HIGH VIB1 then vibrate vibration motor #1 PAUSE 1000 LOW VIB1 ENDIF RETURN sr_sonar2: PULSOUT INIT, 5 PULSIN ECHO,1,wDist wDist = wDist/convfac PAUSE 10 DEBUG DEC wDist, CR PAUSE 20 IF wDist<10 THEN HIGH VIB2 PAUSE 1000 LOW VIB2 ENDIF RETURN sr_sonar3: PULSOUT INIT, 5 PULSIN ECHO,1,wDist wDist = wDist/convfac PAUSE 10 DEBUG DEC wDist, CR PAUSE 20 IF wDist<10 THEN HIGH VIB3 PAUSE 1000 11

LOW VIB3 ENDIF RETURN Read_GP2D12: for IR sensor #1 volts = 0 ' Reset Sensor Value FOR index = 0 TO 2 ' Read 3 Times LOW Adc0831 ' Enable ADC0831 SHIFTIN AdcData, AdcClock, MSBPOST, [result\9] ' Read The Voltage HIGH Adc0831 ' Disable ADC0831 volts = volts + result ' Add The Values PAUSE 30 NEXT volts = volts / 3 ' Average The Readings RETURN Calculate_Distance: FOR index = 0 TO 15 ' Search DATA Table For Value READ (Vout + index), test2 ' Get Value From DATA Table IF (test2 <= volts) THEN EXIT ' Found Value NEXT SELECT index CASE 0 cm = 10 ' Set To Minimum Distance CASE 1 TO 14 ' Calculate Distance cm = 10 + (5 * index) IF (test2 < volts) THEN ' Estimate Using Interpolation READ (Vout + index - 1), test1 slope = (test1 - test2) * 10 / span ' Calculate Slope cm = cm - ((volts - test2) * 10 / slope) ENDIF CASE 15 cm = 80 ' Set To Maximum Distance ENDSELECT IF cm<15 THEN FREQOUT 7, 700, 900

ENDIF RETURN Read_2Y0A02: volts2 = 0 ' Reset second IR Sensor Value FOR index2 = 0 TO 2 ' Read 3 Times LOW Adc0832 ' Enable Second ADC0831 SHIFTIN AdcData2, AdcClock2, MSBPOST, [result2\9] ' Read The Voltage 12

HIGH Adc0832 ' Disable ADC0831 volts2 = volts2 + result2 ' Add The Values PAUSE 30 NEXT volts2 = volts2 / 3 ' Average The Readings RETURN Calculate_Distance_2: FOR index2 = 0 TO 15 ' Search DATA Table For Value READ (Vout2 + index2), test2_2 ' Get Value From DATA Table IF (test2_2 <= volts2) THEN EXIT ' Found Value NEXT SELECT index2 CASE 0 cm2 = 10 ' Set To Minimum Distance CASE 1 TO 14 ' Calculate Distance cm2 = 10 + (5 * index2) IF (test2_2 < volts2) THEN ' Estimate Using Interpolation READ (Vout2 + index2 - 1), test1_2 slope2 = (test1_2 - test2_2) * 10 / span2 ' Calculate Slope cm2 = cm2 - ((volts2 - test2_2) * 10 / slope2) ENDIF CASE 15 cm2 = 80 ' Set To Maximum Distance ENDSELECT IF cm2<15 THEN FREQOUT 7, 700, 1500

ENDIF RETURN

13