US20040267442A1 - Computer-controlled power wheelchair navigation system - Google Patents

Computer-controlled power wheelchair navigation system Download PDF

Info

Publication number
US20040267442A1
US20040267442A1 US10/890,411 US89041104A US2004267442A1 US 20040267442 A1 US20040267442 A1 US 20040267442A1 US 89041104 A US89041104 A US 89041104A US 2004267442 A1 US2004267442 A1 US 2004267442A1
Authority
US
United States
Prior art keywords
wheelchair
computer
path
navigation
navigation system
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
US10/890,411
Other versions
US7383107B2 (en
Inventor
Linda Fehr
Steven Skaar
Guillermo Del Castillo
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
US Department of Veterans Affairs VA
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Priority to US10/890,411 priority Critical patent/US7383107B2/en
Assigned to U.S. DEPARTMENT OF VETERANS AFFAIRS OF EDWARD HINES, JR. HOSPITAL reassignment U.S. DEPARTMENT OF VETERANS AFFAIRS OF EDWARD HINES, JR. HOSPITAL ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: FEHR, LINDA, DEL CASTILLO, GUILLERMO, SKAAR, STEVEN B.
Publication of US20040267442A1 publication Critical patent/US20040267442A1/en
Priority to US12/115,978 priority patent/US20080300777A1/en
Application granted granted Critical
Publication of US7383107B2 publication Critical patent/US7383107B2/en
Adjusted expiration legal-status Critical
Expired - Fee Related legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G01MEASURING; TESTING
    • G01CMEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/20Instruments for performing navigational calculations
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05DSYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
    • G05D1/02Control of position or course in two dimensions
    • G05D1/021Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0231Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
    • G05D1/0246Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means

Definitions

  • This invention relates to a wheelchair navigation system, and more particularly to a computer-controlled power wheelchair navigation system that allows a person to navigate through a location with pre-established paths, and with voice or manually activating the computer.
  • a significant population of severely disabled people require mobility assistance in daily living activities beyond what is available in conventional powered wheelchairs.
  • the present invention is aimed at a segment of that population, those unable to manually guide the movements of powered wheelchairs, to provide them with autonomous navigation capabilities in repetitively used environments such as homes, offices, hospitals, and public buildings and spaces.
  • a robotic wheelchair system commonly known as the WHEELSELSY, developed by the Massachusetts Institute of Technology Artificial Intelligence Laboratory, operates as a semi-autonomous system that is capable of being controlled by head and eye movements of the rider.
  • NavChair® Another wheelchair system known as NavChair® is an adaptive shared control system that is semi-autonomous.
  • the NavChair® uses guidance command from the rider and exerts control in areas such as obstacle avoidance or object approach.
  • the Wheelchair Project also has a system that uses landmark recognition and obstacle avoidance as well as interaction across a spectrum of automations from low level motion guidance to selection of destination wherein the wheelchair delivers the rider.
  • None of these known assistance devices are fully autonomous operations. Moreover, none of these known assistance devices permit avoidance of wheelchair collision with obstacles during navigation of the wheelchair through a taught trajectory while allowing for close approach of the wheelchair to solid bodies present during performance of teaching functions. They do, however, share many common features, including adaptability to a multitude of different vehicles, varying levels of operator control authority, and obstacle avoidance.
  • the autonomous navigation system of the present invention was developed for use in rehabilitative or therapeutic environments as well as domestic and vocational circumstances. It can also be modified for use in independent mobile circumstances for the severely disabled such as public access buildings like museums and airports. Secondary applications for autonomous transport are also foreseeable, although not specifically identified herein.
  • An object of the present invention is to provide a navigation system that will enable independent mobility for handicapped people.
  • Another object of the present invention is to provide a computer-controlled wheelchair that is capable of navigating in various spaces via pre-determined paths of travel, based upon the approach disclosed in the article, “Extending Teach-Repeat to Nonholonomic Robots,” Skaar, S. B. and Yoder, J. D.
  • Yet another object of the present invention is to permit avoidance of wheelchair collision with obstacles during navigation of the wheelchair through a taught trajectory and allowing for close approach of the wheelchair to solid bodies present during performance of teaching functions.
  • the present navigation system utilizes a commercially available powered wheelchair that has been modified to include hardware, encoding devices to provide odometry, video capture devices to provide absolute position information, ultrasonic sensors to detect obstacles, as well as drive and navigation systems.
  • the computer's software has the function of path and destination learning, recognition of obstacles, and navigation to a selected destination.
  • the computer-controlled power wheelchair of the present system is a learning, autonomous, obstruction avoiding, navigation system for motorized wheelchairs.
  • the system learns by being driven over the desired routes to the selected destinations and committing the routes and destination to memory via the computer system. After learning the route, the wheelchair is capable of autonomously navigating any of the learned routes which is selected by the rider.
  • the ‘teach and learn’ system of route establishment is unique within this field of application.
  • the present system is also capable of easily and accurately maneuvering through doorways.
  • the present computer-controlled power wheelchair navigation system provides the physically disabled rider with means of ‘on command’ navigation over frequently used paths.
  • the system was designed to operate either with guidance from the rider or with rider input consisting of a destination command (autonomous operation).
  • This system utilizes any commercially available motorized wheelchair as its base, and is powered by three onboard batteries.
  • a battery charger is also included to keep the batteries charged and operable.
  • Navigation within the system is carried out by use of a known filter-based estimator, such as that by Kalman®.
  • the estimator is used in conjunction with interface software running on an onboard laptop computer. Dual cameras, proximity sensors, microphones, and rotation sensors for the wheels are all mounted to the wheelchair. These electronic components are used with the specialized software in conjunction with small visual markers placed on the walls of the location to be navigated which provide an accurate path over which the wheelchair travels.
  • the wheelchair is ‘taught’ or has pre-programmed paths to various destinations by having an able-bodied person push the chair once, with or without the rider, along any desired route of travel while the computer records information from the sensors. Once destinations have been established, the rider can then select any of these locations and the navigation system will steer the wheelchair to that location. Paths can be reversed and/or merged to reach a desired location, or in order to avoid an obstacle. The system will choose the shortest uninterrupted path to its destination.
  • the computer can be operated by two modes of operation: voice mode and switch mode.
  • voice mode the rider controls the navigation system by speaking commands into a microphone that is attached to the wheelchair and connected to the computer. If the rider cannot address the correct command, they can request a scan of all available commands. A list is simultaneously displayed on the screen and spoken by the text-to-speech converter, when the correct command is selected.
  • switch mode the computer reads the available locations aloud through a speech synthesizer and earphone. When the desired location is spoken, the rider triggers a switch identifying the location. Switching mechanisms have been mounted in various locations on the chair and can be activated by virtually any part of the rider's body.
  • the rider can give commands to the computer controlling the chair at any time during navigation. Such commands can be issued by speaking words or by switching triggers. Some common commands that may be used include: stop, continue travel, return to the last starting point, go faster or slower, and pause the navigation system; however such commands are not limited to these particular commands, but may be customized to fit the rider's needs.
  • the rider When the rider chooses a destination, they will receive feedback from the computer either by a visual display or by synthesized speech.
  • Proximity sensors located on the wheelchair locate obstacles that are in the path of the wheelchair during travel. The sensors trigger the computer to stop the wheelchair if travel is obstructed. The system does not allow the wheelchair to continue to the selected destination until the object is removed, and the wheelchair can be directed to return to a previous point so that it may travel to an alternate destination.
  • the invention achieves the aforementioned exemplary objects by providing a wheelchair navigation system including, a motorized wheelchair, and at least one computer having software that performs navigational functions, teaching functions, obstacle detection, supervisory tasks, and diagnostic analysis.
  • the navigation system may further include at least one camera, at least one proximity sensor, at least one rotation sensor, at least one visual sensor, at least one visual marker, and an input device including a switch or a microphone.
  • the rotation sensor, visual sensor and the input device may be interfaced to the computer which runs specialized software, and the proximity sensor performs sensing operations during performance of the teaching functions for thereby permitting avoidance of wheelchair collision with obstacles during navigation of the wheelchair through a taught trajectory and allowing for close approach of the wheelchair to solid bodies present during performance of the teaching functions.
  • the computer may include a screen with a visual display, and receive information from the cameras and the sensors, process the information and determine a path of travel for the wheelchair from such information.
  • the computer may further be voice or manually activated.
  • the invention yet further provides a wheelchair navigation system including a motorized wheelchair, at least one battery, a battery charger, a filter-based estimator with interface software, a computer having software that performs navigational functions and teaching functions, at least one camera, at least one proximity sensor, at least one microphone, at least one rotation sensor for the wheels, the at least one sensor is mounted on the wheelchair, and a plurality of visual markers capable of being placed on the walls of a specific location.
  • the proximity sensor may perform sensing operations during performance of the teaching functions for thereby permitting avoidance of wheelchair collision with obstacles during navigation of the wheelchair through a taught trajectory and allowing for close approach of the wheelchair to solid bodies present during performance of the teaching functions.
  • the invention also provides method of navigation using a powered wheelchair including the steps of, providing a motorized wheelchair, the wheelchair being powered by at least one onboard battery, and a battery charger and having dual cameras, proximity sensors, a microphone and rotation sensors for sensing wheel rotation, providing a filter-based estimator with interface software running that is operated on at least one computer including a control system, and providing markers at various locations within a specific area.
  • the method may further include the steps of teaching at least one path by walking the wheelchair through the at least one path prior to the wheelchair navigation system being used for independent travel, and recording the at least one path in the computer, sensing the markers with the proximity sensors and the cameras during performance of the teaching for thereby permitting avoidance of wheelchair collision with obstacles during navigation of the wheelchair through a taught trajectory and allowing for close approach of the wheelchair to solid bodies present during performance of the teaching, and providing information about the markers from the proximity sensors and the cameras to the at least one computer.
  • the method may yet further include the steps of operating the software to determine the location of various objects within a location, creating a path for the wheelchair to avoid the objects with the software, recalling the at least one taught path, and directing the wheelchair on the path via the software.
  • the method may include the steps of establishing ultrasound profiles at various junctures throughout a path during performance of the teaching, simplifying the profiles in a postprocessed computer event, and comparing the profiles during tracking through a taught path with current ultrasound sensed profiles.
  • the method may yet further include the step of comparing disparities between the current ultrasound sensed profiles of objects to determine if a configuration of an object close to the taught path has changed since the teaching, and if the configuration has changed by a predetermined threshold, halting the wheelchair.
  • FIG. 1 is a front perspective view of the computer-controlled power wheel chair of the present invention
  • FIG. 2 is a partial front elevation view of the undercarriage of the computer-controlled power wheel chair
  • FIG. 3 is a front elevation view of the computer screen on the computer-controlled power wheel chair
  • FIG. 4 is a top plan view of a sample room showing the various paths
  • FIG. 5 is a top plan view of a sample room showing various paths.
  • FIG. 6 is a schematic chart of parallel user-interface processes.
  • the computer-controlled power wheelchair navigation 10 is comprised of a wheelchair 20 with the traditional amenities such as a seat 30 , a back support portion 40 , arms 50 , a head rest 60 , elevating leg-rests 71 including foot rests 70 and leg supports 80 , front wheels 90 , rear wheels 100 , rear storage space 110 , and an undercarriage 120 .
  • the wheelchair 20 is powered by a power supply such as batteries 130 .
  • a power supply such as batteries 130 .
  • a chin switch 140 shown in FIG. 1 there are a variety of means available to control the wheelchair, such as a chin switch 140 shown in FIG. 1, however other conventional control means such as joysticks, buttons, voice control and etc, all known in the art, may be incorporated into the present system.
  • the wheelchair navigation system 10 also has at least one computer 150 that is seated in a computer docking station within the rear storage space 110 .
  • At least one video camera 160 is connected to the computer 150 and aids in identifying the location of the wheelchair 20 when in use. Any of the video cameras 160 detect cues 170 that are strategically placed throughout a given location.
  • Input to the navigation system's estimation algorithm in the computer 150 is provided by at least one digital shaft encoder 180 mounted at the wheelchair's drive wheel to sense wheel rotation, and by at least one monochrome video camera which detects wall-mounted cues 170 via video 160 .
  • the cues 170 provide information to the computer 150 so that the computer 150 can identify the location of the wheelchair 20 and provide a pre-programmed or taught path for the wheelchair 20 to follow.
  • the computer-controlled wheelchair navigation system 10 relies on a unique ‘teach-repeat’ paradigm of control in which a person manually pushes the wheelchair 20 along each path to be included in the system's 10 repertoire. New destinations or routes of travel made obsolete by changes in the environment are easily ‘retaught’ in the same way.
  • the system uses an estimator that is based on a Kalman® filter. The estimator is activated throughout the pre-programming or teaching episodes, and a compact record of the position and orientation histories of each trajectory is stored in the computer 150 , establishing a reference path for each desired destination. In subsequent autonomous navigation, the wheelchair navigation system 10 chooses the point toward which to steer based on the geometric relationship between the filter's current estimate of the actual position of the wheelchair 20 and the previously ‘learned’ reference path.
  • inputs as well as the output voltage to the wheelchair controller 190 are interfaced with the computer 150 . All user input to the system is via a standard keyboard (not shown). Ultrasonic proximity sensors 162 are interfaced to a second computer 152 (not shown) which analyzes the time rate of change in the distance of near objects so as to assess the likelihood of a collision. The second computer 152 generates a digital proceed/halt signal serially interfaced to the full-sized navigation computer 150 . By continuously polling this signal, the navigation system 10 can bring the wheelchair 20 smoothly to a rest to avert collision with an obstacle, resuming travel along the reference path once the obstacle is removed.
  • the base for the navigation system is a commonly known wheelchair 20 .
  • the wheelchair 20 can be adjustable and such amenities will not interfere with the wheelchair navigation system 10 .
  • a pneumatic back support system 40 , adjustable headrest 60 , and elevating leg supports 80 may also be added to the wheelchair 20 to provide maximum flexibility with respect to riders with limited postural stability.
  • the wheelchair's undercarriage 120 provides the support base for the navigation system components.
  • the undercarriage 120 accommodates the video cameras or sensors 160 which are mounted below the wheelchair seat 30 and positioned in such a way that their view is unobstructed by the rider when they are seated in the wheelchair 20 .
  • the undercarriage 120 also houses, in addition to at least one wheelchair battery 130 , a separate battery 132 and a DC/AC inverter 200 for powering the components of the navigation system 10 .
  • there are two 24 volt wheelchair batteries 130 with an additional 12 volt separate battery 132 , however it is understood that various types and sizes of other batteries known in the art may be used within this system.
  • a second embodiment of the navigation system uses a single computer 150 .
  • the computer 150 must accommodate interfacing with a variety of hardware components.
  • the second embodiment also has digital encoders 180 and upgraded video cameras or sensors 160 .
  • the second embodiment uses PCMCIA cards for capturing digital encoder signals, and for providing the output voltage to the wheelchair controller 190 . Since no suitable ‘frame grabber’ board having either a PCMCIA or USB interface can be found, the present system incorporates a full-sized PCI frame grabber, and interfaces it with the computer 150 via a docking station 210 . Video cameras 160 are serially interfaced to the computer 150 .
  • Navigation and obstacle detection software modules use various types of programming language such as C and C++, but are not limited thereto.
  • the software module is also designed to run as ‘threads’ or processes controlled by the user interface software.
  • the navigation software is extensively modified to reflect the newly-modeled system dynamics of the wheelchair 20 as well as the upgraded components such as cameras and PC cards.
  • User interface software such as Windows-95 based visual C++ programming is used in the present embodiment, however it is foreseeable that other comparable software can be used.
  • the software enables the rider to control the navigation system 10 by speaking commands and/or switch activation. The determination of whether to commence navigation in voice or switch mode is made when the main software module is launched.
  • a dialog window 230 governs which mode the navigation system 10 is going to operate in.
  • Other characteristics of the rider's ensuing navigation session is input via a number of checked boxes, radio buttons, and a slider control shown in the dialog window 230 .
  • the values of these controls are saved to disk in the computer 150 from session to session so that if the rider is using the chair with the same configuration of options as they used previously, an able-bodied individual may launch the main software module, ensure the rider has access to their switch and/or microphone, position the wheelchair 20 in such a way that it can detect its initialization cues 170 , and start the program. From this point forward, no further intervention on behalf of an able bodied non-rider is required other than perhaps to shut the system down after use and recharge the batteries, if such is not accessible to the rider.
  • the rider issues a command not to travel, the travel is canceled leaving the rider free to select another destination.
  • the rider may stop the travel of the wheelchair 20 by issuing a command to discontinue travel, such as to wait or stop.
  • These commands interrupt the wheelchair's motion (without confirmation), and the chair 20 remains stationary until a command is issued for the wheelchair 20 to proceed.
  • the rider is informed when the wheelchair 20 arrives at its final destination (or retraces its path completely back to its starting point) with a message that the travel has been completed.
  • Such commands can be either audio commands or switch activated commands.
  • the rider wears either a headset or an earplug to facilitate aural scanning (not shown).
  • a text-to-speech (TTS) engine sequentially ‘speaks’ a list of available destinations or commands.
  • TTS text-to-speech
  • the rider activates the switch, the most recently articulated destination or command is selected.
  • the speed with which the synthesized voice scans the rider's options maybe adjusted at any time during the navigation session by lengthening or shortening the pause separating each utterance.
  • the scanning rate is set by a slider control 240 on the initial dialog screen 230 (FIG. 3).
  • This dialog window 230 also contains an options section wherein there is a box designated for scanning after a specified number of utterances 250 as well as another box designated to allow for sleep during travel 260 of the wheelchair 20 . If the sleep during travel box is checked, scanning is suspended once the rider chooses and confirms a destination, and motion commences. Otherwise, scanning is continuous during travel. In either case, the rider may suspend scanning at any time by issuing a command to shut the navigation system 10 down.
  • Any switch 140 may be used as an input mechanism for the device. Examples include but are not limited to: a bite switch (a small plastic pipette interfaced to a pneumatic switch), a sip-puff switch, a “twitch” switch, or any “universal” switch (stick, pillow, pneumatic cushion, etc.) which can be activated by virtually any part of the body.
  • a bite switch a small plastic pipette interfaced to a pneumatic switch
  • sip-puff switch a sip-puff switch
  • a “twitch” switch or any “universal” switch (stick, pillow, pneumatic cushion, etc.) which can be activated by virtually any part of the body.
  • These switches 140 can be mounted to any part of the wheelchair 20 and may be swapped in and out of the system via a small modularized electronics or switch input box 220 (FIG. 1) interfaced to the parallel port of the computer 150 .
  • the rider wears a headset with a small microphone (not shown) which are commonly known in the art.
  • a voice recognition computer program also commercially available, is used to facilitate the spoken commands.
  • the rider Prior to operating the navigation system by voice for the first time, the rider must “train” the speech recognition software to recognize their individual voice. Nevertheless, the navigation system user interface software provides for optional supplementary switch input to handle recognition errors. For example, if the check box “Scan after 4 unrecognized utterances” 250 is left unchecked on the setup dialog window 230 (FIG. 3), unrecognized speech, such as conversation, is ignored by the navigation system 10 .
  • the system will respond to each unrecognized utterance with either a voice then synthesized message or a typed response that the command is unclear. Then the computer 150 will either prompt the rider for another command, or wait until another command is given. After a series of unrecognized utterances (i.e. three if the number of utterances is set for four), the software can “read” to the rider a list of available destinations. After yet another a unrecognized utterance (i.e. the fourth), the software enters scanning mode and the rider makes their selection via switch activation. After a successful selection by switch, the software returns to voice mode.
  • a series of unrecognized utterances i.e. three if the number of utterances is set for four
  • the software can “read” to the rider a list of available destinations.
  • the software enters scanning mode and the rider makes their selection via switch activation. After a successful selection by switch, the software returns to voice
  • the rider has the option to suspend voice recognition any time by rendering a command for the system to close down.
  • the navigation system user interface software will only respond to a pre-selected activation command (i.e. the spoken word “Navigate”), which reactivates the voice recognition.
  • a pre-selected activation command i.e. the spoken word “Navigate”
  • the rider may elicit an aural listing of the available choices at any time by asking for their options.
  • the rider is also capable of changing between the voice and scanning modes. This is facilitated by another options box 290 , labeled “Allow mode changes” on the setup dialog screen (FIG. 3). Therefore, as conditions vary the rider can direct the navigation system either verbally by spoken words or by switch activation. This feature is optional since not all riders will be able to use both modes. In other cases, the rider may have a preference for a given mode but may choose to change modes when they encounter changing conditions. The rider changes operating modes in the same way that other commands are issued: by manually selecting the specified mode while scanning, or by speaking a designated phrase identifying the scanning mode while in voice mode. The rider may move between operating modes as often as desired.
  • the rider may use the dialog window 230 as an aid in identifying the various destinations known to the navigation system 10 by viewing a box 310 listing a tree structure list of the various destinations.
  • Names at the root level may represent separate environments (i.e. home).
  • the next level of destinations may represent rooms or other areas of interest within the given environment (i.e. the kitchen), with the lowest level representing specific locations within rooms or areas (i.e. the sink or stove).
  • initial position estimates were either entered manually from the keyboard or were read from the first path file to be executed. This demanded exact placement of the wheelchair 20 at these coordinates prior to commencing travel, an inconvenient and potentially difficult task with a rider in the chair. Therefore, an additional software module can be used to estimate the wheelchair's initial position based on each camera's 160 ability to locate a unique configuration of cues 170 . To enable autonomous navigation for the rider, the wheelchair 20 need only be placed anywhere in the vicinity of these cues 170 when the main software module is launched.
  • One of the first tasks of the locator module is to call the initialization routine in a loop. If either camera 160 cannot detect its initialization cues 170 , the caregiver/attendant is notified accordingly by both a synthesized voice message and a message box (not shown). The message instructs the rider to verify that the cameras 160 are turned on, to ascertain that the cues 170 are not obstructed, and if necessary, to reposition the wheelchair 20 . Having done so, the caregiver/attendant merely touches a key to repeat initialization.
  • the rider may be possible for the rider to control velocity of the wheelchair 20 with commands. However, it may also be controlled automatically by the navigation algorithm which slows the chair when position estimate errors are high, so that it may process more video information per inch traveled. The navigation algorithm may increase speed gradually as estimates improve.
  • the rider also has the ability to stop the wheelchair 20 completely and then either continue along the selected path or retrace the path back toward its starting point.
  • the rider may suspend the voice activated user interface to prevent speech recognition from being confusing, or so that those in the conversation will not be distracted by continued aural scanning.
  • the wheelchair 20 may remain stopped for any length of time, and may continue along the selected path simply by the rider issuing a command to continue.
  • the rider may issue the command to go back, and the wheelchair will commence retracing its path.
  • the rider may change their decision of travel to a particular destination at any time and return to the point of departure.
  • the command to go back or return is also useful if the rider has issued a command to stop late in the process, and has actually passed the point where they wished to stop. After issuing the command to go back, the rider may back up to the desired stopping point, issue a command to stop again, and then continue at the rider's discretion.
  • halt command is intended only for the rider's convenience and operating functionality. It is recognized that any rider in any particular situation may or may not have the ability to stop the wheelchair 20 quickly enough to avoid an obstacle.
  • the chair's proximity sensors 162 are intended for this purpose.
  • the original navigation software of the first embodiment is provided for execution of only one pre-programmed path at a time.
  • the updated software of the second embodiment if no pre-programmed path exists between the rider's current location and the requested destination, the system seeks to construct a path by joining first two, then three, then four or five pre-programmed paths. The shortest possible path is constructed, and the process is seamless to the rider.
  • the navigation system also has a utility that takes an existing pre-programmed path and creates a new path with all segments reversed.
  • This utility and the path joining capability are of significant benefit to the rider and their caregiver since it is no longer necessary to teach or pre-program a specific path from each location in the environment to every other location in the environment.
  • the utility becomes increasingly important as the number of locations increases since the number of possible paths connecting n destinations is n*(n-1) (i.e. 90 paths for 10 destinations).
  • the path utility is only appropriate in certain circumstances since the rider does not generally wish to travel in reverse.
  • the placement of furniture and fixtures together with the need for a specific terminal orientation of the wheelchair 20 may dictate that the most expedient route of travel is in the reverse. This is particularly true when distances are short.
  • the best route from the desk D to the window W may be the reverse of the path from the window W to the desk D.
  • the chair is at the couch C, there may be insufficient space to turn the wheelchair 20 around so the best path to the window W is in reverse, and the most expedient path to the desk D is to back up to the window W and proceed to the desk D using the path already taught (path W-D from the window W to the desk D).
  • a person other than the rider may teach or pre-program paths to an intermediate node X which is central to four desired locations W,D,C,T. With path joining, all 12 possible paths between these four locations may be realized using only the 8 pre-programmed or taught paths shown, X-T, T-X, X—C, C—X, X-D, D-X, X—W, and W—X.
  • the first embodiment has navigation software running on a computer 150 interfaced with a second computer 152 .
  • the second computer 152 runs obstacle detection software. All user input to the system is done via a keyboard.
  • the second embodiment has a single computer 150 .
  • the main module has a user-interface (U/I) thread which, after all setup and initialization functions have been performed, the thread monitors the microphone or switch for user commands.
  • U/I thread initiates the navigation thread which, in turn, initiates the sonar thread for obstacle detection.
  • the U/I thread continues to monitor the microphone or switch in the event the rider wishes to stop the wheelchair 20 .
  • the navigation and sonar threads are suspended until the rider requests further travel. Under these circumstances, resources are consumed only as needed, ensuring maximum responsiveness of the user interface to the rider's actions.
  • the threads communicate with each other by setting and reading a status variable.
  • FIG. 6 represents a schematic chart of these parallel processes.
  • the navigation system 10 can navigate through an area using a single camera 160 .
  • the system's position estimates are increasingly uncertain, which causes the wheelchair 20 to reduce speed. This could occur at precisely the moment more power is needed to cross from one location into another (i.e. from linoleum to carpet).
  • a brief burst of power is delivered to the wheelchair controller 190 , enabling the wheelchair 20 to overcome its inertia. This refinement allows the navigation system to reliably transition from one location to another location which may have a different type of surface (e.g. tile vs. carpet).
  • Errors in the system's position estimates are calculated from video images in units of pixels. If an error is within a given tolerance, the location of the identified cue 170 is incorporated into the algorithm's estimate of the chair's position. If not, the cue 170 is rejected and the position estimate incorporates only odometry information (dead reckoning). Since the cameras 160 are close to the cues 170 in certain situations such as a narrow hallway, errors in this region are magnified to such an extent that the wheelchair 20 is forced to dead reckon even when cues 170 are properly identified. The generalized solution to this situation is to associate dynamic error tolerances to individual path segments within the files that define each route of travel.
  • Initialization cues 170 are strategically placed within a specific location wherein all travel within that location commences from that starting point. A variety of paths are pre-programmed in this location.
  • the current navigation system is a vision-based system in which video detection of wall-mounted cues 170 is combined with odometery information to obtain accurate ongoing estimates of the wheelchair's position and orientation.
  • the navigation software sends to the power wheelchair controller 190 a signal simulating that of a joystick and guides the wheelchair 20 accurately along paths which the system has previously been pre-programmed or taught by an able bodied human instructor.
  • the ‘teach-repeat’ paradigm of control in which human teacher manually pushes the wheelchair 20 along any desired route of travel, obviates the need to fully characterize the geometry of the environment. Such characterizations, normally required for automatic trajectory planners, are difficult to acquire and maintain.
  • the navigation system's 10 only use of ultrasound is to detect near obstacles with which it might otherwise collide.
  • the navigation system provides fully autonomous navigation. That is, the principal responsibility for steering is delegated to the navigation system 10 with the rider retaining the ability to, at their own discretion, halt and resume travel, or halt and retrace the current path toward the point of departure.
  • the paths are pre-programmed or taught by a human instructor, they readily incorporate close approach to furniture, passage between objects with minimal clearance, and complex motions such as the multiple changes of direction required to maneuver into tight spaces.
  • the human judgment inherent in the original teaching episode is brought to bear in each repetition of the taught path.
  • the present system extends existing theoretical development for making use of a sequence of incoming, imperfect information combined with a nominal, also imperfect, differential equation model of the ‘plant’ or system, to produce ongoing best estimates of the ‘state’ of the system.
  • a numerical-integration-based filter is used, and described in detail in the article “An Autonomous Vision-Based Mobile Robot,” by E. T. Baumgartner and S. B. Skaar.
  • the navigation system 10 dispenses with time, the usual independent variable, and uses instead a kinematic independent variable: the average forward rotation of the two actuated drive wheels 100 of the wheelchair 20 .
  • the vehicle will not ‘cut corners’ which could jeopardize rider safety in order to “catch-up” with the time-based reference.
  • the rider may choose to assert manual control of the wheelchair 20 at any time, taking it from the actual reference path. Since the system's estimator within the computer 150 is activated whether or not it is in control of the chair's motion, and since the control loop is time independent, the navigation system 10 is capable of re-engaging the target path without resorting to any special estimation or control algorithm even though an unplanned detour takes the wheelchair 20 some distance away.
  • Wheel rotation measurements are taken by the digital shaft encoder 180 and supplemented by observation of visual cues 170 placed in convenient locations throughout the environment of interest.
  • Use of these cues 170 offers a major advantage over the more commonly used ultrasound or sonar in that the ‘signature’ of the image is distinct, and the frequency of corroborating input is high; hence, there is negligible danger that the system's estimator will become ‘confused.’
  • any number of navigation vehicles 20 may navigate the same environment, referencing the same cues 170 , without interfering with each other.
  • the rider may control the navigation system by a single, discrete input such as a spoken word, activation of any simple switch, or touchscreen input.
  • the system provides feedback to the rider via synthesized speech, visual cues, or aural cue.
  • destinations known to the navigation system are stored in a data structure such that only the destinations reachable from a given location are available when the rider is at that location. Once the rider selects an available destination and confirms their intention to travel, motion along the chosen path commences.
  • the rider may use their input device to assert such supervisory functions as influencing travel speed, halting motion at any point, resuming the original reference path, changing destinations, retracing the path to return to the point of departure, or reasserting manual control of the wheelchair, to the extent the rider is able.
  • ultrasonic proximity sensors 162 may be used as described below. Specifically, as described above, sensors 162 may be interfaced to second computer 152 (not shown) which analyzes the time rate of change in the distance of near objects so as to assess the likelihood of a collision. The second computer 152 generates a digital proceed/halt signal serially interfaced to the full-sized navigation computer 150 . By continuously polling this signal, navigation system 10 can bring wheelchair 20 smoothly to a rest to avert collision with an obstacle, resuming travel along the reference path once the obstacle is removed.
  • ultrasound sensing may be applied to avoid collisions with obstacles while at the same time allowing for the close approach to solid bodies as part of the trajectory objective (e.g. approaching an appliance in the kitchen or bathroom, or approaching a bed.)
  • each-repeat may entail the use of complementary ultrasound sensing by means of sensors 162 in an important and essential way, for many real-world applications. Unlike any prior art teach-repeat strategies or the teach-repeat strategy disclosed above, ultrasound sensors 162 may be operated during teaching, not simply during tracking, in order to avoid obstacles. While the existing disclosure of Steven B.
  • the aforementioned feature of ultrasound sensing during teaching is therefore particularly enabling when the task requires very close approach to a solid object such as when nearing a bathroom fixture. Moreover, the aforementioned feature of ultrasound sensing during teaching is by definition a strategy unavailable outside of the teach/repeat approach to navigation.
  • ultrasound sensors 162 may be used to avoid obstacles as wheelchair 20 proceeds through its taught trajectory. Obstacles, including humans or animals, may be introduced in the path of wheelchair 20 after teaching and thereby interfere with execution of a taught path. Ultrasound sensors are inexpensive proximity sensors capable of automatically determining the distance between the sensor and the closest solid object. Given that wheelchair navigation system 10 as part of its operation keeps track of the current coordinates of wheelchair 20 , including those when the ultrasound pulse is emitted, and given that the path being executed has already been taught and is therefore known to the controlling computer, it would follow that signals from sensors 162 would be sufficient to compute whether continued execution of the path should result in collision. However, properties associated with ultrasound do not support this simple conclusion.
  • Ultrasound sensors 162 may be placed at any locations on wheelchair 20 . This results in a virtual shield for the rider without the impractical consequence of being unable to execute the wide range of potentially crucial motions that entail close approach to various solid bodies such as sinks, toilets, furniture, doors and so on.

Abstract

A wheelchair navigation system for a motorized wheelchair includes dual cameras, proximity sensors, microphones, and rotation sensors for the wheels. Small markers are placed on the walls of a location or room. The navigation system uses the proximity sensors, rotation sensors and cameras in conjunction with the specialized software to determine where objects or impediments are located in the room and thereby redirect the path of the wheelchair so as to avoid such objects. The wheelchair is walked through the marked location thereby ‘teaching’ various paths which are recorded in the computer and recalled later when the wheelchair is in use. The proximity sensor perform sensing operations during performance of the teaching functions for thereby permitting avoidance of wheelchair collision with obstacles during navigation of the wheelchair through a taught trajectory and allowing for close approach of the wheelchair to solid bodies present during performance of the teaching functions.

Description

    RELATED APPLICATIONS
  • This application is a continuation-in-part of application Ser. No. 10/187,989, filed Jul. 2, 2002, currently pending.[0001]
  • [0002] The U.S. Government has a paid-up license in this invention and the right in limited circumstances to require the patent owner to license others on reasonable terms as provided for by the terms of Contract No. ______ (or Grant No. ______) awarded by the Veterans Affairs Agency ______.
  • BACKGROUND OF INVENTION
  • a. Field of Invention [0003]
  • This invention relates to a wheelchair navigation system, and more particularly to a computer-controlled power wheelchair navigation system that allows a person to navigate through a location with pre-established paths, and with voice or manually activating the computer. [0004]
  • b. Description of Related Art [0005]
  • A significant population of severely disabled people require mobility assistance in daily living activities beyond what is available in conventional powered wheelchairs. The present invention is aimed at a segment of that population, those unable to manually guide the movements of powered wheelchairs, to provide them with autonomous navigation capabilities in repetitively used environments such as homes, offices, hospitals, and public buildings and spaces. [0006]
  • There are a number of navigational methods that already exist for autonomous and semi-autonomous vehicles. Current navigational systems for semi-autonomous vehicles usually have controls such as a joystick or an ocular device, and employ overriding features such as obstacle or collision avoidance. Some autonomous systems utilize path-following methods such as a magnetic strip, optical strip or targets to provide paths from an origin to a destination. These methods have difficulty handling multiple and intersecting paths, if they can handle them at all. [0007]
  • There are three systems specifically identified in the prior art, all of which differ from the present invention. A robotic wheelchair system, commonly known as the WHEELSELSY, developed by the Massachusetts Institute of Technology Artificial Intelligence Laboratory, operates as a semi-autonomous system that is capable of being controlled by head and eye movements of the rider. [0008]
  • Another wheelchair system known as NavChair® is an adaptive shared control system that is semi-autonomous. The NavChair® uses guidance command from the rider and exerts control in areas such as obstacle avoidance or object approach. [0009]
  • The Wheelchair Project also has a system that uses landmark recognition and obstacle avoidance as well as interaction across a spectrum of automations from low level motion guidance to selection of destination wherein the wheelchair delivers the rider. [0010]
  • None of these known assistance devices are fully autonomous operations. Moreover, none of these known assistance devices permit avoidance of wheelchair collision with obstacles during navigation of the wheelchair through a taught trajectory while allowing for close approach of the wheelchair to solid bodies present during performance of teaching functions. They do, however, share many common features, including adaptability to a multitude of different vehicles, varying levels of operator control authority, and obstacle avoidance. [0011]
  • The autonomous navigation system of the present invention was developed for use in rehabilitative or therapeutic environments as well as domestic and vocational circumstances. It can also be modified for use in independent mobile circumstances for the severely disabled such as public access buildings like museums and airports. Secondary applications for autonomous transport are also foreseeable, although not specifically identified herein. [0012]
  • SUMMARY OF INVENTION
  • An object of the present invention is to provide a navigation system that will enable independent mobility for handicapped people. [0013]
  • Another object of the present invention is to provide a computer-controlled wheelchair that is capable of navigating in various spaces via pre-determined paths of travel, based upon the approach disclosed in the article, “Extending Teach-Repeat to Nonholonomic Robots,” Skaar, S. B. and Yoder, J. D. [0014]
  • Yet another object of the present invention is to permit avoidance of wheelchair collision with obstacles during navigation of the wheelchair through a taught trajectory and allowing for close approach of the wheelchair to solid bodies present during performance of teaching functions. [0015]
  • The present navigation system utilizes a commercially available powered wheelchair that has been modified to include hardware, encoding devices to provide odometry, video capture devices to provide absolute position information, ultrasonic sensors to detect obstacles, as well as drive and navigation systems. The computer's software has the function of path and destination learning, recognition of obstacles, and navigation to a selected destination. [0016]
  • The computer-controlled power wheelchair of the present system is a learning, autonomous, obstruction avoiding, navigation system for motorized wheelchairs. The system learns by being driven over the desired routes to the selected destinations and committing the routes and destination to memory via the computer system. After learning the route, the wheelchair is capable of autonomously navigating any of the learned routes which is selected by the rider. The ‘teach and learn’ system of route establishment is unique within this field of application. The present system is also capable of easily and accurately maneuvering through doorways. [0017]
  • The present computer-controlled power wheelchair navigation system provides the physically disabled rider with means of ‘on command’ navigation over frequently used paths. The system was designed to operate either with guidance from the rider or with rider input consisting of a destination command (autonomous operation). This system utilizes any commercially available motorized wheelchair as its base, and is powered by three onboard batteries. A battery charger is also included to keep the batteries charged and operable. [0018]
  • Navigation within the system is carried out by use of a known filter-based estimator, such as that by Kalman®. The estimator is used in conjunction with interface software running on an onboard laptop computer. Dual cameras, proximity sensors, microphones, and rotation sensors for the wheels are all mounted to the wheelchair. These electronic components are used with the specialized software in conjunction with small visual markers placed on the walls of the location to be navigated which provide an accurate path over which the wheelchair travels. [0019]
  • The wheelchair is ‘taught’ or has pre-programmed paths to various destinations by having an able-bodied person push the chair once, with or without the rider, along any desired route of travel while the computer records information from the sensors. Once destinations have been established, the rider can then select any of these locations and the navigation system will steer the wheelchair to that location. Paths can be reversed and/or merged to reach a desired location, or in order to avoid an obstacle. The system will choose the shortest uninterrupted path to its destination. [0020]
  • The computer can be operated by two modes of operation: voice mode and switch mode. In the voice mode, the rider controls the navigation system by speaking commands into a microphone that is attached to the wheelchair and connected to the computer. If the rider cannot address the correct command, they can request a scan of all available commands. A list is simultaneously displayed on the screen and spoken by the text-to-speech converter, when the correct command is selected. [0021]
  • In switch mode, the computer reads the available locations aloud through a speech synthesizer and earphone. When the desired location is spoken, the rider triggers a switch identifying the location. Switching mechanisms have been mounted in various locations on the chair and can be activated by virtually any part of the rider's body. [0022]
  • The rider can give commands to the computer controlling the chair at any time during navigation. Such commands can be issued by speaking words or by switching triggers. Some common commands that may be used include: stop, continue travel, return to the last starting point, go faster or slower, and pause the navigation system; however such commands are not limited to these particular commands, but may be customized to fit the rider's needs. [0023]
  • When the rider chooses a destination, they will receive feedback from the computer either by a visual display or by synthesized speech. Proximity sensors located on the wheelchair locate obstacles that are in the path of the wheelchair during travel. The sensors trigger the computer to stop the wheelchair if travel is obstructed. The system does not allow the wheelchair to continue to the selected destination until the object is removed, and the wheelchair can be directed to return to a previous point so that it may travel to an alternate destination. [0024]
  • With respect to alternative navigation technologies, fixed track systems pose numerous disadvantages: difficult and inflexible implementation, undue complexity as the number of intersecting paths grows larger, and inability of the wheelchair to depart from and return to the prescribed path without outside intervention. Use of sonar for point-to-point navigation presents such problems as specular reflection (spurious measurements obtained when sonar ‘bounces’ off multiple surfaces) and the likelihood of multiple chairs intercepting each other's navigation signals in institutional settings. [0025]
  • In contrast, multiple chairs fitted with the vision-based navigation technology may readily share wall-mounted cues without difficulty, and the navigation system's use of sonar to locate obstacles within 2-3 feet of the chair is significantly less challenging than employing sonar for navigation. Finally, some researchers are investigating autonomous navigation via video detection of ‘natural landmarks’, i.e. locating and steering toward a door by identifying a doorknob. In comparison, the present navigation system relies on images with a distinctive ‘signature.’ This lends itself to significantly simpler and more straightforward implementation with negligible danger of the estimator becoming ‘confused’. Of course, the navigation system technology is inherently limited to modified environments, those defined by cue placement and taught paths. [0026]
  • Other objects, advantages and novel features of the invention will become more apparent from the following detailed description of the invention when taken in conjunction with the accompanying drawings. [0027]
  • The invention achieves the aforementioned exemplary objects by providing a wheelchair navigation system including, a motorized wheelchair, and at least one computer having software that performs navigational functions, teaching functions, obstacle detection, supervisory tasks, and diagnostic analysis. The navigation system may further include at least one camera, at least one proximity sensor, at least one rotation sensor, at least one visual sensor, at least one visual marker, and an input device including a switch or a microphone. The rotation sensor, visual sensor and the input device may be interfaced to the computer which runs specialized software, and the proximity sensor performs sensing operations during performance of the teaching functions for thereby permitting avoidance of wheelchair collision with obstacles during navigation of the wheelchair through a taught trajectory and allowing for close approach of the wheelchair to solid bodies present during performance of the teaching functions. [0028]
  • For the wheelchair navigation system described above, the computer may include a screen with a visual display, and receive information from the cameras and the sensors, process the information and determine a path of travel for the wheelchair from such information. The computer may further be voice or manually activated. [0029]
  • The invention yet further provides a wheelchair navigation system including a motorized wheelchair, at least one battery, a battery charger, a filter-based estimator with interface software, a computer having software that performs navigational functions and teaching functions, at least one camera, at least one proximity sensor, at least one microphone, at least one rotation sensor for the wheels, the at least one sensor is mounted on the wheelchair, and a plurality of visual markers capable of being placed on the walls of a specific location. The proximity sensor may perform sensing operations during performance of the teaching functions for thereby permitting avoidance of wheelchair collision with obstacles during navigation of the wheelchair through a taught trajectory and allowing for close approach of the wheelchair to solid bodies present during performance of the teaching functions. [0030]
  • The invention also provides method of navigation using a powered wheelchair including the steps of, providing a motorized wheelchair, the wheelchair being powered by at least one onboard battery, and a battery charger and having dual cameras, proximity sensors, a microphone and rotation sensors for sensing wheel rotation, providing a filter-based estimator with interface software running that is operated on at least one computer including a control system, and providing markers at various locations within a specific area. The method may further include the steps of teaching at least one path by walking the wheelchair through the at least one path prior to the wheelchair navigation system being used for independent travel, and recording the at least one path in the computer, sensing the markers with the proximity sensors and the cameras during performance of the teaching for thereby permitting avoidance of wheelchair collision with obstacles during navigation of the wheelchair through a taught trajectory and allowing for close approach of the wheelchair to solid bodies present during performance of the teaching, and providing information about the markers from the proximity sensors and the cameras to the at least one computer. The method may yet further include the steps of operating the software to determine the location of various objects within a location, creating a path for the wheelchair to avoid the objects with the software, recalling the at least one taught path, and directing the wheelchair on the path via the software. [0031]
  • For the method described above, the method may include the steps of establishing ultrasound profiles at various junctures throughout a path during performance of the teaching, simplifying the profiles in a postprocessed computer event, and comparing the profiles during tracking through a taught path with current ultrasound sensed profiles. The method may yet further include the step of comparing disparities between the current ultrasound sensed profiles of objects to determine if a configuration of an object close to the taught path has changed since the teaching, and if the configuration has changed by a predetermined threshold, halting the wheelchair. [0032]
  • Additional features, advantages, and embodiments of the invention may be set forth or apparent from consideration of the following detailed description, drawings, and claims. Moreover, it is to be understood that both the foregoing summary of the invention and the following detailed description are exemplary and intended to provide further explanation without limiting the scope of the invention as claimed.[0033]
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate preferred embodiments of the invention and together with the detail description serve to explain the principles of the invention. In the drawings: [0034]
  • FIG. 1 is a front perspective view of the computer-controlled power wheel chair of the present invention; [0035]
  • FIG. 2 is a partial front elevation view of the undercarriage of the computer-controlled power wheel chair; [0036]
  • FIG. 3 is a front elevation view of the computer screen on the computer-controlled power wheel chair; [0037]
  • FIG. 4 is a top plan view of a sample room showing the various paths; [0038]
  • FIG. 5 is a top plan view of a sample room showing various paths; and [0039]
  • FIG. 6 is a schematic chart of parallel user-interface processes.[0040]
  • DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
  • The computer-controlled [0041] power wheelchair navigation 10 is comprised of a wheelchair 20 with the traditional amenities such as a seat 30, a back support portion 40, arms 50, a head rest 60, elevating leg-rests 71 including foot rests 70 and leg supports 80, front wheels 90, rear wheels 100, rear storage space 110, and an undercarriage 120. The wheelchair 20 is powered by a power supply such as batteries 130. There are a variety of means available to control the wheelchair, such as a chin switch 140 shown in FIG. 1, however other conventional control means such as joysticks, buttons, voice control and etc, all known in the art, may be incorporated into the present system.
  • The [0042] wheelchair navigation system 10 also has at least one computer 150 that is seated in a computer docking station within the rear storage space 110. At least one video camera 160 is connected to the computer 150 and aids in identifying the location of the wheelchair 20 when in use. Any of the video cameras 160 detect cues 170 that are strategically placed throughout a given location. Input to the navigation system's estimation algorithm in the computer 150 is provided by at least one digital shaft encoder 180 mounted at the wheelchair's drive wheel to sense wheel rotation, and by at least one monochrome video camera which detects wall-mounted cues 170 via video 160. The cues 170 provide information to the computer 150 so that the computer 150 can identify the location of the wheelchair 20 and provide a pre-programmed or taught path for the wheelchair 20 to follow.
  • The basic components of a filter and a differential equation model of nonlinear dynamics of the [0043] wheelchair 20 combined with a sequence of incoming sensor information, wheel rotations and video images, produce ongoing best estimates of the position and orientation or ‘state’ of the wheelchair 20 in the two-dimensional space of the floor.
  • The computer-controlled [0044] wheelchair navigation system 10 relies on a unique ‘teach-repeat’ paradigm of control in which a person manually pushes the wheelchair 20 along each path to be included in the system's 10 repertoire. New destinations or routes of travel made obsolete by changes in the environment are easily ‘retaught’ in the same way. The system uses an estimator that is based on a Kalman® filter. The estimator is activated throughout the pre-programming or teaching episodes, and a compact record of the position and orientation histories of each trajectory is stored in the computer 150, establishing a reference path for each desired destination. In subsequent autonomous navigation, the wheelchair navigation system 10 chooses the point toward which to steer based on the geometric relationship between the filter's current estimate of the actual position of the wheelchair 20 and the previously ‘learned’ reference path.
  • In one embodiment, inputs as well as the output voltage to the [0045] wheelchair controller 190 are interfaced with the computer 150. All user input to the system is via a standard keyboard (not shown). Ultrasonic proximity sensors 162 are interfaced to a second computer 152 (not shown) which analyzes the time rate of change in the distance of near objects so as to assess the likelihood of a collision. The second computer 152 generates a digital proceed/halt signal serially interfaced to the full-sized navigation computer 150. By continuously polling this signal, the navigation system 10 can bring the wheelchair 20 smoothly to a rest to avert collision with an obstacle, resuming travel along the reference path once the obstacle is removed.
  • The base for the navigation system is a commonly known [0046] wheelchair 20. The wheelchair 20 can be adjustable and such amenities will not interfere with the wheelchair navigation system 10. A pneumatic back support system 40, adjustable headrest 60, and elevating leg supports 80 may also be added to the wheelchair 20 to provide maximum flexibility with respect to riders with limited postural stability.
  • The wheelchair's [0047] undercarriage 120, as shown in FIG. 2, provides the support base for the navigation system components. The undercarriage 120 accommodates the video cameras or sensors 160 which are mounted below the wheelchair seat 30 and positioned in such a way that their view is unobstructed by the rider when they are seated in the wheelchair 20. The undercarriage 120 also houses, in addition to at least one wheelchair battery 130, a separate battery 132 and a DC/AC inverter 200 for powering the components of the navigation system 10. In this particular embodiment of the wheelchair navigation system, there are two 24 volt wheelchair batteries 130, with an additional 12 volt separate battery 132, however it is understood that various types and sizes of other batteries known in the art may be used within this system.
  • A second embodiment of the navigation system uses a [0048] single computer 150. The computer 150 must accommodate interfacing with a variety of hardware components. The second embodiment also has digital encoders 180 and upgraded video cameras or sensors 160. The second embodiment uses PCMCIA cards for capturing digital encoder signals, and for providing the output voltage to the wheelchair controller 190. Since no suitable ‘frame grabber’ board having either a PCMCIA or USB interface can be found, the present system incorporates a full-sized PCI frame grabber, and interfaces it with the computer 150 via a docking station 210. Video cameras 160 are serially interfaced to the computer 150.
  • Navigation and obstacle detection software modules use various types of programming language such as C and C++, but are not limited thereto. The software module is also designed to run as ‘threads’ or processes controlled by the user interface software. At the same time, the navigation software is extensively modified to reflect the newly-modeled system dynamics of the [0049] wheelchair 20 as well as the upgraded components such as cameras and PC cards.
  • User interface software, such as Windows-95 based visual C++ programming is used in the present embodiment, however it is foreseeable that other comparable software can be used. The software enables the rider to control the [0050] navigation system 10 by speaking commands and/or switch activation. The determination of whether to commence navigation in voice or switch mode is made when the main software module is launched.
  • A dialog window [0051] 230, shown in FIG. 3, governs which mode the navigation system 10 is going to operate in. Other characteristics of the rider's ensuing navigation session is input via a number of checked boxes, radio buttons, and a slider control shown in the dialog window 230. The values of these controls are saved to disk in the computer 150 from session to session so that if the rider is using the chair with the same configuration of options as they used previously, an able-bodied individual may launch the main software module, ensure the rider has access to their switch and/or microphone, position the wheelchair 20 in such a way that it can detect its initialization cues 170, and start the program. From this point forward, no further intervention on behalf of an able bodied non-rider is required other than perhaps to shut the system down after use and recharge the batteries, if such is not accessible to the rider.
  • In both voice and switch modes, feedback to the rider is provided by synthesized speech. For example, if the rider requests travel to a desk as a location, the computer will inquire if the rider is ready to engage the path ‘Desk’. The rider must always confirm their intention to travel with a specific command, either spoken or manually input, at which time motion along the chosen path commences. [0052]
  • If the rider issues a command not to travel, the travel is canceled leaving the rider free to select another destination. During travel, the rider may stop the travel of the [0053] wheelchair 20 by issuing a command to discontinue travel, such as to wait or stop. These commands interrupt the wheelchair's motion (without confirmation), and the chair 20 remains stationary until a command is issued for the wheelchair 20 to proceed. The rider is informed when the wheelchair 20 arrives at its final destination (or retraces its path completely back to its starting point) with a message that the travel has been completed. Such commands can be either audio commands or switch activated commands.
  • In the switch input mode, the rider wears either a headset or an earplug to facilitate aural scanning (not shown). In this mode, a text-to-speech (TTS) engine sequentially ‘speaks’ a list of available destinations or commands. When the rider activates the switch, the most recently articulated destination or command is selected. The speed with which the synthesized voice scans the rider's options maybe adjusted at any time during the navigation session by lengthening or shortening the pause separating each utterance. The scanning rate is set by a [0054] slider control 240 on the initial dialog screen 230 (FIG. 3). This dialog window 230 also contains an options section wherein there is a box designated for scanning after a specified number of utterances 250 as well as another box designated to allow for sleep during travel 260 of the wheelchair 20. If the sleep during travel box is checked, scanning is suspended once the rider chooses and confirms a destination, and motion commences. Otherwise, scanning is continuous during travel. In either case, the rider may suspend scanning at any time by issuing a command to shut the navigation system 10 down.
  • Once navigation has been suspended, either by explicit command or because travel has commenced, scanning can be resumed at any time by activating the switch. Any [0055] switch 140 may be used as an input mechanism for the device. Examples include but are not limited to: a bite switch (a small plastic pipette interfaced to a pneumatic switch), a sip-puff switch, a “twitch” switch, or any “universal” switch (stick, pillow, pneumatic cushion, etc.) which can be activated by virtually any part of the body. These switches 140 can be mounted to any part of the wheelchair 20 and may be swapped in and out of the system via a small modularized electronics or switch input box 220 (FIG. 1) interfaced to the parallel port of the computer 150.
  • In voice mode, the rider wears a headset with a small microphone (not shown) which are commonly known in the art. A voice recognition computer program, also commercially available, is used to facilitate the spoken commands. Prior to operating the navigation system by voice for the first time, the rider must “train” the speech recognition software to recognize their individual voice. Nevertheless, the navigation system user interface software provides for optional supplementary switch input to handle recognition errors. For example, if the check box “Scan after 4 unrecognized utterances” [0056] 250 is left unchecked on the setup dialog window 230 (FIG. 3), unrecognized speech, such as conversation, is ignored by the navigation system 10. If this box 250 is checked, the system will respond to each unrecognized utterance with either a voice then synthesized message or a typed response that the command is unclear. Then the computer 150 will either prompt the rider for another command, or wait until another command is given. After a series of unrecognized utterances (i.e. three if the number of utterances is set for four), the software can “read” to the rider a list of available destinations. After yet another a unrecognized utterance (i.e. the fourth), the software enters scanning mode and the rider makes their selection via switch activation. After a successful selection by switch, the software returns to voice mode. Regardless of the status of the options box 250,260, the rider has the option to suspend voice recognition any time by rendering a command for the system to close down. Once suspended, the navigation system user interface software will only respond to a pre-selected activation command (i.e. the spoken word “Navigate”), which reactivates the voice recognition. As a final feature in voice mode, the rider may elicit an aural listing of the available choices at any time by asking for their options.
  • The rider is also capable of changing between the voice and scanning modes. This is facilitated by another [0057] options box 290, labeled “Allow mode changes” on the setup dialog screen (FIG. 3). Therefore, as conditions vary the rider can direct the navigation system either verbally by spoken words or by switch activation. This feature is optional since not all riders will be able to use both modes. In other cases, the rider may have a preference for a given mode but may choose to change modes when they encounter changing conditions. The rider changes operating modes in the same way that other commands are issued: by manually selecting the specified mode while scanning, or by speaking a designated phrase identifying the scanning mode while in voice mode. The rider may move between operating modes as often as desired.
  • The rider may use the dialog window [0058] 230 as an aid in identifying the various destinations known to the navigation system 10 by viewing a box 310 listing a tree structure list of the various destinations. Names at the root level may represent separate environments (i.e. home). The next level of destinations may represent rooms or other areas of interest within the given environment (i.e. the kitchen), with the lowest level representing specific locations within rooms or areas (i.e. the sink or stove).
  • Using this [0059] tree structure 310, only destinations reachable from a given location are available when the rider is at that location. For example, if the rider is in the kitchen at home, the elevator at work is not an available destination. This type of organization produces the shortest possible list of options to scan in switch mode, and minimizes recognition errors in voice mode by limiting the active vocabulary to a logical subset of possible destinations. It also allows use of destination names that would otherwise be ambiguous. For example, there may be a table in the rider's kitchen as well as in their bedroom. Knowledge of the rider's current location resolves the ambiguity. If paths have been taught in multiple environments, the environment for the ensuing navigation session may be chosen from the drop-down box 320.
  • With the first embodiment, initial position estimates were either entered manually from the keyboard or were read from the first path file to be executed. This demanded exact placement of the [0060] wheelchair 20 at these coordinates prior to commencing travel, an inconvenient and potentially difficult task with a rider in the chair. Therefore, an additional software module can be used to estimate the wheelchair's initial position based on each camera's 160 ability to locate a unique configuration of cues 170. To enable autonomous navigation for the rider, the wheelchair 20 need only be placed anywhere in the vicinity of these cues 170 when the main software module is launched.
  • One of the first tasks of the locator module is to call the initialization routine in a loop. If either [0061] camera 160 cannot detect its initialization cues 170, the caregiver/attendant is notified accordingly by both a synthesized voice message and a message box (not shown). The message instructs the rider to verify that the cameras 160 are turned on, to ascertain that the cues 170 are not obstructed, and if necessary, to reposition the wheelchair 20. Having done so, the caregiver/attendant merely touches a key to repeat initialization.
  • It may be possible for the rider to control velocity of the [0062] wheelchair 20 with commands. However, it may also be controlled automatically by the navigation algorithm which slows the chair when position estimate errors are high, so that it may process more video information per inch traveled. The navigation algorithm may increase speed gradually as estimates improve.
  • The rider also has the ability to stop the [0063] wheelchair 20 completely and then either continue along the selected path or retrace the path back toward its starting point.
  • If a conversation takes place within the vicinity of the [0064] wheelchair 20, the rider may suspend the voice activated user interface to prevent speech recognition from being confusing, or so that those in the conversation will not be distracted by continued aural scanning. The wheelchair 20 may remain stopped for any length of time, and may continue along the selected path simply by the rider issuing a command to continue.
  • Alternatively, the rider may issue the command to go back, and the wheelchair will commence retracing its path. Hence, the rider may change their decision of travel to a particular destination at any time and return to the point of departure. The command to go back or return is also useful if the rider has issued a command to stop late in the process, and has actually passed the point where they wished to stop. After issuing the command to go back, the rider may back up to the desired stopping point, issue a command to stop again, and then continue at the rider's discretion. This combination of features, in effect, provides a virtually infinite number of destinations along taught paths. [0065]
  • It should be noted that the halt command is intended only for the rider's convenience and operating functionality. It is recognized that any rider in any particular situation may or may not have the ability to stop the [0066] wheelchair 20 quickly enough to avoid an obstacle. The chair's proximity sensors 162 are intended for this purpose.
  • The original navigation software of the first embodiment is provided for execution of only one pre-programmed path at a time. With the updated software of the second embodiment, if no pre-programmed path exists between the rider's current location and the requested destination, the system seeks to construct a path by joining first two, then three, then four or five pre-programmed paths. The shortest possible path is constructed, and the process is seamless to the rider. [0067]
  • The navigation system also has a utility that takes an existing pre-programmed path and creates a new path with all segments reversed. This utility and the path joining capability are of significant benefit to the rider and their caregiver since it is no longer necessary to teach or pre-program a specific path from each location in the environment to every other location in the environment. The utility becomes increasingly important as the number of locations increases since the number of possible paths connecting n destinations is n*(n-1) (i.e. 90 paths for 10 destinations). [0068]
  • The path utility is only appropriate in certain circumstances since the rider does not generally wish to travel in reverse. However, the placement of furniture and fixtures together with the need for a specific terminal orientation of the [0069] wheelchair 20 may dictate that the most expedient route of travel is in the reverse. This is particularly true when distances are short. For example, in FIG. 4, the best route from the desk D to the window W may be the reverse of the path from the window W to the desk D. Similarly, if the chair is at the couch C, there may be insufficient space to turn the wheelchair 20 around so the best path to the window W is in reverse, and the most expedient path to the desk D is to back up to the window W and proceed to the desk D using the path already taught (path W-D from the window W to the desk D).
  • If all these conditions were true, the six possible paths connecting all three destinations W,D,C are implemented by teaching only the two paths shown, path W-D and path W-C. Even when the path reversal utility is not appropriate, the path joining capability may significantly reduce teaching time and effort, depending on the geometry of the environment and the fact that paths may be taught to intermediate locations which are not true destinations of interest. [0070]
  • As shown in FIG. 5, a person other than the rider may teach or pre-program paths to an intermediate node X which is central to four desired locations W,D,C,T. With path joining, all 12 possible paths between these four locations may be realized using only the 8 pre-programmed or taught paths shown, X-T, T-X, X—C, C—X, X-D, D-X, X—W, and W—X. [0071]
  • As previously discussed, the first embodiment has navigation software running on a [0072] computer 150 interfaced with a second computer 152. The second computer 152 runs obstacle detection software. All user input to the system is done via a keyboard.
  • The second embodiment has a [0073] single computer 150. The main module has a user-interface (U/I) thread which, after all setup and initialization functions have been performed, the thread monitors the microphone or switch for user commands. Once the rider has selected and confirmed a destination for travel, the U/I thread initiates the navigation thread which, in turn, initiates the sonar thread for obstacle detection. During travel, the U/I thread continues to monitor the microphone or switch in the event the rider wishes to stop the wheelchair 20. When travel is completed, the navigation and sonar threads are suspended until the rider requests further travel. Under these circumstances, resources are consumed only as needed, ensuring maximum responsiveness of the user interface to the rider's actions. The threads communicate with each other by setting and reading a status variable. FIG. 6 represents a schematic chart of these parallel processes.
  • The [0074] navigation system 10 can navigate through an area using a single camera 160. However, by using a single camera 160, the system's position estimates are increasingly uncertain, which causes the wheelchair 20 to reduce speed. This could occur at precisely the moment more power is needed to cross from one location into another (i.e. from linoleum to carpet). To solve this problem, when the wheelchair 20 fails to produce nonzero wheel rotations for a specified number (i.e. 200) of consecutive processing cycles, a brief burst of power is delivered to the wheelchair controller 190, enabling the wheelchair 20 to overcome its inertia. This refinement allows the navigation system to reliably transition from one location to another location which may have a different type of surface (e.g. tile vs. carpet).
  • Errors in the system's position estimates are calculated from video images in units of pixels. If an error is within a given tolerance, the location of the identified [0075] cue 170 is incorporated into the algorithm's estimate of the chair's position. If not, the cue 170 is rejected and the position estimate incorporates only odometry information (dead reckoning). Since the cameras 160 are close to the cues 170 in certain situations such as a narrow hallway, errors in this region are magnified to such an extent that the wheelchair 20 is forced to dead reckon even when cues 170 are properly identified. The generalized solution to this situation is to associate dynamic error tolerances to individual path segments within the files that define each route of travel.
  • [0076] Initialization cues 170 are strategically placed within a specific location wherein all travel within that location commences from that starting point. A variety of paths are pre-programmed in this location.
  • The current navigation system is a vision-based system in which video detection of wall-mounted [0077] cues 170 is combined with odometery information to obtain accurate ongoing estimates of the wheelchair's position and orientation.
  • The navigation software sends to the power wheelchair controller [0078] 190 a signal simulating that of a joystick and guides the wheelchair 20 accurately along paths which the system has previously been pre-programmed or taught by an able bodied human instructor. The ‘teach-repeat’ paradigm of control, in which human teacher manually pushes the wheelchair 20 along any desired route of travel, obviates the need to fully characterize the geometry of the environment. Such characterizations, normally required for automatic trajectory planners, are difficult to acquire and maintain. The navigation system's 10 only use of ultrasound is to detect near obstacles with which it might otherwise collide.
  • The navigation system provides fully autonomous navigation. That is, the principal responsibility for steering is delegated to the [0079] navigation system 10 with the rider retaining the ability to, at their own discretion, halt and resume travel, or halt and retrace the current path toward the point of departure.
  • Because the paths are pre-programmed or taught by a human instructor, they readily incorporate close approach to furniture, passage between objects with minimal clearance, and complex motions such as the multiple changes of direction required to maneuver into tight spaces. The human judgment inherent in the original teaching episode is brought to bear in each repetition of the taught path. [0080]
  • The present system extends existing theoretical development for making use of a sequence of incoming, imperfect information combined with a nominal, also imperfect, differential equation model of the ‘plant’ or system, to produce ongoing best estimates of the ‘state’ of the system. For nonlinear system dynamics such as those of a motorized wheelchair, a numerical-integration-based filter is used, and described in detail in the article “An Autonomous Vision-Based Mobile Robot,” by E. T. Baumgartner and S. B. Skaar. Because only nominal kinematic rather than kinetic equations are used, the [0081] navigation system 10 dispenses with time, the usual independent variable, and uses instead a kinematic independent variable: the average forward rotation of the two actuated drive wheels 100 of the wheelchair 20. Since time is no longer the independent variable for control, new means of advancing forward along the reference path must be devised. As with human control of vehicles, it has proven possible and natural to advance the ‘target’ juncture on the reference path via estimated location rather than time. That is, the point along the path toward which to steer is chosen based on the geometric relationship between the current estimate of actual position and the previously ‘learned’ reference path.
  • Several advantages attend the use of forward average wheel rotation as the independent variable. First, since the differential equations are now time-independent, it becomes convenient to control the speed of the vehicle entirely independently of the estimation and tracking algorithms. Hence, vehicle speed may be altered in response to any set of conditions without affecting the position-estimation or tracking algorithm. [0082]
  • Secondly, the vehicle will not ‘cut corners’ which could jeopardize rider safety in order to “catch-up” with the time-based reference. [0083]
  • Finally, the rider may choose to assert manual control of the [0084] wheelchair 20 at any time, taking it from the actual reference path. Since the system's estimator within the computer 150 is activated whether or not it is in control of the chair's motion, and since the control loop is time independent, the navigation system 10 is capable of re-engaging the target path without resorting to any special estimation or control algorithm even though an unplanned detour takes the wheelchair 20 some distance away.
  • Wheel rotation measurements are taken by the [0085] digital shaft encoder 180 and supplemented by observation of visual cues 170 placed in convenient locations throughout the environment of interest. Use of these cues 170 offers a major advantage over the more commonly used ultrasound or sonar in that the ‘signature’ of the image is distinct, and the frequency of corroborating input is high; hence, there is negligible danger that the system's estimator will become ‘confused.’ Also, any number of navigation vehicles 20 may navigate the same environment, referencing the same cues 170, without interfering with each other.
  • The rider may control the navigation system by a single, discrete input such as a spoken word, activation of any simple switch, or touchscreen input. The system provides feedback to the rider via synthesized speech, visual cues, or aural cue. To facilitate path selection, destinations known to the navigation system are stored in a data structure such that only the destinations reachable from a given location are available when the rider is at that location. Once the rider selects an available destination and confirms their intention to travel, motion along the chosen path commences. During travel, the rider may use their input device to assert such supervisory functions as influencing travel speed, halting motion at any point, resuming the original reference path, changing destinations, retracing the path to return to the point of departure, or reasserting manual control of the wheelchair, to the extent the rider is able. [0086]
  • In order to avoid unforeseen obstacles that (by definition) were not there when the tracked trajectory was taught, [0087] ultrasonic proximity sensors 162 may be used as described below. Specifically, as described above, sensors 162 may be interfaced to second computer 152 (not shown) which analyzes the time rate of change in the distance of near objects so as to assess the likelihood of a collision. The second computer 152 generates a digital proceed/halt signal serially interfaced to the full-sized navigation computer 150. By continuously polling this signal, navigation system 10 can bring wheelchair 20 smoothly to a rest to avert collision with an obstacle, resuming travel along the reference path once the obstacle is removed.
  • In order to further utilize [0088] sensors 162, enabling use can be made of the record of ultrasound readings acquired during trajectory teaching in order to permit discrimination between actual obstacles and solid objects that are simply in place (were in place during teaching) but that are not a concern with respect to the trajectory currently being tracked.
  • In this manner, ultrasound sensing may be applied to avoid collisions with obstacles while at the same time allowing for the close approach to solid bodies as part of the trajectory objective (e.g. approaching an appliance in the kitchen or bathroom, or approaching a bed.) [0089]
  • In this regard, in addition to the aforementioned vision-based repeat strategy, the “teach-repeat” disclosed herein may entail the use of complementary ultrasound sensing by means of [0090] sensors 162 in an important and essential way, for many real-world applications. Unlike any prior art teach-repeat strategies or the teach-repeat strategy disclosed above, ultrasound sensors 162 may be operated during teaching, not simply during tracking, in order to avoid obstacles. While the existing disclosure of Steven B. Skaar “Extending Teach-Repeat to Nonholomonic Robots,” Structronic Systems, 1998, 316-342, does mention using proximity sensing including ultrasound as a supplement to vision for pose assessment during both teaching and tracking, this is not the implication of the disclosure herein, and indeed use of the ultrasound data during teaching serves a very different purpose from this: Ultrasound signals from sensors 162 are well known to create imprecise representations of the current surroundings due to such sensor attributes as specular reflection. However, given similar sensor/object juxtapositions, signals from ultrasound sensors 162 are highly repeatable. Use of sensors 162 during teaching, then, is an excellent way to ensure that in the immediate vicinity of travel objects before wheelchair 20 during a tracking or “repeat” event are as they were when teaching occurred.
  • The aforementioned feature of ultrasound sensing during teaching is therefore particularly enabling when the task requires very close approach to a solid object such as when nearing a bathroom fixture. Moreover, the aforementioned feature of ultrasound sensing during teaching is by definition a strategy unavailable outside of the teach/repeat approach to navigation. [0091]
  • Based upon the aforementioned, during motion of [0092] wheelchair 20, ultrasound sensors 162 may be used to avoid obstacles as wheelchair 20 proceeds through its taught trajectory. Obstacles, including humans or animals, may be introduced in the path of wheelchair 20 after teaching and thereby interfere with execution of a taught path. Ultrasound sensors are inexpensive proximity sensors capable of automatically determining the distance between the sensor and the closest solid object. Given that wheelchair navigation system 10 as part of its operation keeps track of the current coordinates of wheelchair 20, including those when the ultrasound pulse is emitted, and given that the path being executed has already been taught and is therefore known to the controlling computer, it would follow that signals from sensors 162 would be sufficient to compute whether continued execution of the path should result in collision. However, properties associated with ultrasound do not support this simple conclusion. In practice, the well-known specular reflections of ultrasound echoes, together with the progressive spread of the emitted sound reduce the accuracy of any such calculation. Combined with the fact that many of the practical paths or trajectories of interest may require close proximity to solid objects this fact precludes this simple use of sensors 162.
  • Nevertheless, there is a synergy between the teach/repeat operation of the basic navigation capability and the application of ultrasound for effective obstacle avoidance. This synergy is based upon the fact that ultrasound responses where sensors are similarly placed with respect to solid objects of the same geometry are highly repeatable. As a consequence, with regard to the aforementioned teach-repeat strategy, there is the prospect of using the teaching event to establish ultrasound profiles at various junctures throughout the current path. These profiles are simplified in a postprocessed computer event and compared during tracking with the current ultrasound profiles. It is significant disparities or discrepancies between the local (to the path segment in question) profiles that becomes the criterion for determining whether or not it is likely that the configuration of objects close to the taught path have changed significantly since the teaching event. [0093]
  • If there has been such a change, and if the aforementioned calculation indicates a nearing of near-future path to any such new object, then the probability of contact of [0094] wheelchair 20 with the object can be regarded as high. Such a means of tolerating very close proximity to unchanged objects while avoiding new, introduced bodies is critically dependent upon the teach/repeat mode of navigation. Additionally, in the event that autonomous system 10 determines that the probability of contact is too high to continue, another capability of teach/repeat can be drawn upon: Once system 10 halts forward progress along the path in order to prevent contact, it is then able to identically reverse its progress along the portion of the path executed up until the point of coming to rest. This is essentially a reversal of all previously executed motions and its feasibility is guaranteed provided, over the short time period since the current tracking event began, no additional obstacles were introduced to block that recently executed portion.
  • [0095] Ultrasound sensors 162, provided their location is permanent, may be placed at any locations on wheelchair 20. This results in a virtual shield for the rider without the impractical consequence of being unable to execute the wide range of potentially crucial motions that entail close approach to various solid bodies such as sinks, toilets, furniture, doors and so on.
  • Although particular embodiments of the invention have been described in detail herein with reference to the accompanying drawings, it is to be understood that the invention is not limited to those particular embodiments, and that various changes and modifications may be effected therein by one skilled in the art without departing from the scope or spirit of the invention as defined in the appended claims. [0096]

Claims (16)

What is claimed is:
1. A wheelchair navigation system comprising:
a motorized wheelchair;
at least one computer having software that performs
navigational functions,
teaching functions,
obstacle detection,
supervisory tasks, and
diagnostic analysis;
at least one camera;
at least one proximity sensor;
at least one rotation sensor;
at least one visual sensor;
at least one visual marker; and
an input device including one of a switch and a microphone,
wherein said at least one rotation sensor, said at least one visual sensor and said input device are interfaced to said at least one computer which runs specialized software, and said at least one proximity sensor performs sensing operations during performance of said teaching functions for thereby permitting avoidance of wheelchair collision with obstacles during navigation of said wheelchair through a taught trajectory and allowing for close approach of said wheelchair to solid bodies present during performance of said teaching functions.
2. A wheelchair navigation system according to claim 1, wherein:
said at least one computer has a screen with a visual display.
3. A wheelchair navigation system according to claim 1, wherein:
said computer receives information from said cameras and said sensors, processes said information and determines a path of travel for said wheelchair from such information.
4. A wheelchair navigation system according to claim 1, wherein:
said computer is voice activated.
5. A wheelchair navigation system according to claim 1, wherein:
said computer is manually operated.
6. A wheelchair navigation system comprising:
a motorized wheelchair;
at least one battery;
a battery charger;
a filter-based estimator with interface software;
a computer having software that performs navigational functions and teaching functions;
at least one camera;
at least one proximity sensor;
at least one microphone;
at least one rotation sensor for the wheels, said at least one sensor is mounted on the wheelchair; and
a plurality of visual markers capable of being placed on the walls of a specific location
wherein said at least one proximity sensor performs sensing operations during performance of said teaching functions for thereby permitting avoidance of wheelchair collision with obstacles during navigation of said wheelchair through a taught trajectory and allowing for close approach of said wheelchair to solid bodies present during performance of said teaching functions.
7. A wheelchair navigation system according to claim 6, wherein:
said at least one computer has a screen with a visual display that is visible to a person using the wheelchair.
8. A wheelchair navigation system according to claim 6, wherein:
said computer receives information from said cameras and said sensors, processes said information and determines a path of travel for said wheelchair from such information.
9. A wheelchair navigation system according to claim 6, wherein:
said computer is voice activated.
10. A wheelchair navigation system according to claim 6, wherein:
said computer is manually operated.
11. A method of navigation using a powered wheelchair comprising the steps of:
providing a motorized wheelchair, said wheelchair being powered by at least one onboard battery, and a battery charger and having dual cameras, proximity sensors, a microphone and rotation sensors for sensing wheel rotation;
providing a filter-based estimator with interface software running that is operated on at least one computer including a control system;
providing markers at various locations within a specific area;
teaching at least one path by walking said wheelchair through said at least one path prior to said wheelchair navigation system being used for independent travel, and recording said at least one path in said computer;
sensing said markers with said proximity sensors and said cameras during performance of said teaching for thereby permitting avoidance of wheelchair collision with obstacles during navigation of said wheelchair through a taught trajectory and allowing for close approach of said wheelchair to solid bodies present during performance of said teaching;
providing information about said markers from said proximity sensors and said cameras to said at least one computer;
operating said software to determine the location of various objects within a location;
creating a path for said wheelchair to avoid said objects with said software;
recalling said at least one taught path; and
directing said wheelchair on said path via said software.
12. A method of navigation using a powered wheelchair according to claim 11, further comprising the step of:
operating said computer via voice control.
13. A method of navigation using a powered wheelchair according to claim 11, further comprising the step of:
operating said computer manually.
14. A method of navigation using a powered wheelchair according to claim 11, further comprising the step of:
establishing ultrasound profiles at various junctures throughout a path during performance of said teaching.
15. A method of navigation using a powered wheelchair according to claim 14, further comprising the step of:
simplifying said profiles in a postprocessed computer event; and
comparing said profiles during tracking through a taught path with current ultrasound sensed profiles.
16. A method of navigation using a powered wheelchair according to claim 15, further comprising the step of:
comparing disparities between said current ultrasound sensed profiles of objects to determine if a configuration of an object close to the taught path has changed since said teaching, and if said configuration has changed by a predetermined threshold, halting said wheelchair.
US10/890,411 2002-07-02 2004-07-14 Computer-controlled power wheelchair navigation system Expired - Fee Related US7383107B2 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US10/890,411 US7383107B2 (en) 2002-07-02 2004-07-14 Computer-controlled power wheelchair navigation system
US12/115,978 US20080300777A1 (en) 2002-07-02 2008-05-06 Computer-controlled power wheelchair navigation system

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US10/187,989 US6842692B2 (en) 2002-07-02 2002-07-02 Computer-controlled power wheelchair navigation system
US10/890,411 US7383107B2 (en) 2002-07-02 2004-07-14 Computer-controlled power wheelchair navigation system

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US10/187,989 Continuation-In-Part US6842692B2 (en) 2002-07-02 2002-07-02 Computer-controlled power wheelchair navigation system

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US12/115,978 Continuation-In-Part US20080300777A1 (en) 2002-07-02 2008-05-06 Computer-controlled power wheelchair navigation system

Publications (2)

Publication Number Publication Date
US20040267442A1 true US20040267442A1 (en) 2004-12-30
US7383107B2 US7383107B2 (en) 2008-06-03

Family

ID=29999432

Family Applications (2)

Application Number Title Priority Date Filing Date
US10/187,989 Expired - Lifetime US6842692B2 (en) 2002-07-02 2002-07-02 Computer-controlled power wheelchair navigation system
US10/890,411 Expired - Fee Related US7383107B2 (en) 2002-07-02 2004-07-14 Computer-controlled power wheelchair navigation system

Family Applications Before (1)

Application Number Title Priority Date Filing Date
US10/187,989 Expired - Lifetime US6842692B2 (en) 2002-07-02 2002-07-02 Computer-controlled power wheelchair navigation system

Country Status (3)

Country Link
US (2) US6842692B2 (en)
AU (1) AU2003281326A1 (en)
WO (1) WO2004005852A1 (en)

Cited By (34)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2009046282A1 (en) * 2007-10-03 2009-04-09 University Of Delaware Et Al. Intelligent powered mobility for infants and special needs children
US7594556B1 (en) * 2004-08-27 2009-09-29 Cook Technologies, Inc. System for storing and retrieving a personal-transportation vehicle
US20100123574A1 (en) * 2008-11-14 2010-05-20 Honda Motor Co., Ltd. Monitoring system for vehicle
US20120136666A1 (en) * 2010-11-29 2012-05-31 Corpier Greg L Automated personal assistance system
WO2014011992A2 (en) * 2012-07-13 2014-01-16 Love Park Robotics, Llc Drive-control systems for vehicles such as personal-transportation vehicles
US8731738B2 (en) 2010-08-31 2014-05-20 University Of Delaware Powered mobility systems and methods
US9316502B2 (en) * 2014-07-22 2016-04-19 Toyota Motor Engineering & Manufacturing North America, Inc. Intelligent mobility aid device and method of navigating and providing assistance to a user thereof
USD768024S1 (en) 2014-09-22 2016-10-04 Toyota Motor Engineering & Manufacturing North America, Inc. Necklace with a built in guidance device
US9488482B2 (en) 2013-08-30 2016-11-08 Elwha Llc Systems and methods for adjusting a contour of a vehicle based on a protrusion
US9578307B2 (en) 2014-01-14 2017-02-21 Toyota Motor Engineering & Manufacturing North America, Inc. Smart necklace with stereo vision and onboard processing
US9576460B2 (en) 2015-01-21 2017-02-21 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable smart device for hazard detection and warning based on image and audio data
US9586318B2 (en) 2015-02-27 2017-03-07 Toyota Motor Engineering & Manufacturing North America, Inc. Modular robot with smart device
US9629774B2 (en) 2014-01-14 2017-04-25 Toyota Motor Engineering & Manufacturing North America, Inc. Smart necklace with stereo vision and onboard processing
US9677901B2 (en) 2015-03-10 2017-06-13 Toyota Motor Engineering & Manufacturing North America, Inc. System and method for providing navigation instructions at optimal times
US9757054B2 (en) 2013-08-30 2017-09-12 Elwha Llc Systems and methods for warning of a protruding body part of a wheelchair occupant
US9811752B2 (en) 2015-03-10 2017-11-07 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable smart device and method for redundant object identification
US9898039B2 (en) 2015-08-03 2018-02-20 Toyota Motor Engineering & Manufacturing North America, Inc. Modular smart necklace
US9915545B2 (en) 2014-01-14 2018-03-13 Toyota Motor Engineering & Manufacturing North America, Inc. Smart necklace with stereo vision and onboard processing
US9922236B2 (en) 2014-09-17 2018-03-20 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable eyeglasses for providing social and environmental awareness
US9919694B1 (en) 2014-04-28 2018-03-20 Ball Aerospace & Technologies Corp. Control system for a complex mechanical system
US9958275B2 (en) 2016-05-31 2018-05-01 Toyota Motor Engineering & Manufacturing North America, Inc. System and method for wearable smart device communications
US9972216B2 (en) 2015-03-20 2018-05-15 Toyota Motor Engineering & Manufacturing North America, Inc. System and method for storing and playback of information for blind users
US10012505B2 (en) 2016-11-11 2018-07-03 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable system for providing walking directions
US10024680B2 (en) 2016-03-11 2018-07-17 Toyota Motor Engineering & Manufacturing North America, Inc. Step based guidance system
US10024667B2 (en) 2014-08-01 2018-07-17 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable earpiece for providing social and environmental awareness
US10024679B2 (en) 2014-01-14 2018-07-17 Toyota Motor Engineering & Manufacturing North America, Inc. Smart necklace with stereo vision and onboard processing
US10024678B2 (en) 2014-09-17 2018-07-17 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable clip for providing social and environmental awareness
US10172760B2 (en) 2017-01-19 2019-01-08 Jennifer Hendrix Responsive route guidance and identification system
US10248856B2 (en) 2014-01-14 2019-04-02 Toyota Motor Engineering & Manufacturing North America, Inc. Smart necklace with stereo vision and onboard processing
US10360907B2 (en) 2014-01-14 2019-07-23 Toyota Motor Engineering & Manufacturing North America, Inc. Smart necklace with stereo vision and onboard processing
US10432851B2 (en) 2016-10-28 2019-10-01 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable computing device for detecting photography
US10490102B2 (en) 2015-02-10 2019-11-26 Toyota Motor Engineering & Manufacturing North America, Inc. System and method for braille assistance
US10521669B2 (en) 2016-11-14 2019-12-31 Toyota Motor Engineering & Manufacturing North America, Inc. System and method for providing guidance or feedback to a user
US10561519B2 (en) 2016-07-20 2020-02-18 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable computing device having a curved back to reduce pressure on vertebrae

Families Citing this family (67)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7188056B2 (en) * 2002-09-09 2007-03-06 Maia Institute Method and apparatus of simulating movement of an autonomous entity through an environment
US20040220735A1 (en) * 2003-04-29 2004-11-04 Adams Don L. Powered mobility vehicle collision damage prevention device
US7862287B2 (en) * 2004-04-13 2011-01-04 Egan Thomas F Multi-motion lifting and transferring apparatus and method
JP4679254B2 (en) * 2004-10-28 2011-04-27 富士通株式会社 Dialog system, dialog method, and computer program
US7403844B2 (en) * 2005-08-31 2008-07-22 Invacare Corporation Method and apparatus for programming parameters of a power driven wheelchair for a plurality of drive settings
US20070050111A1 (en) * 2005-08-31 2007-03-01 Invacare Corp. Method and apparatus for automated positioning of user support surfaces in power driven wheelchair
US20070050096A1 (en) * 2005-08-31 2007-03-01 Invacare Corporation Programmable actuator controller for power positioning seat or leg support of a wheelchair
PT103354B (en) * 2005-09-21 2007-07-12 Univ Do Minho CONTROL SYSTEM FOR OMNIDIRECTIONAL WHEEL CHAIRS
ES2296542B2 (en) * 2006-10-09 2009-02-01 Universidad De Malaga ROBOTIZED WHEELCHAIR WITH AUTONOMOUS OPERATING CAPACITY.
US7719213B2 (en) 2006-10-19 2010-05-18 Herman Stephen A Door actuator and opener
US8781725B2 (en) * 2006-12-21 2014-07-15 Verizon Patent And Licensing Inc. Methods and apparatus for capability-specific routing
DE102006062061B4 (en) * 2006-12-29 2010-06-10 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Apparatus, method and computer program for determining a position based on a camera image from a camera
JP4916900B2 (en) * 2007-01-25 2012-04-18 富士通コンポーネント株式会社 Direction presenting system, and electric wheelchair, cane and game controller to which the direction presenting system is applied
US20080228384A1 (en) * 2007-03-17 2008-09-18 Erickson Clinton W Navigational system for a personal mobility device
US7625314B2 (en) * 2007-04-30 2009-12-01 Nike, Inc. Adaptive training system with aerial mobility system
DE102007048263A1 (en) 2007-10-08 2009-04-09 Robert Bosch Gmbh Navigation device for use in e.g. motor-driven wheelchair, has processing unit determining operating range and/or time frame for mobile user based on ergo data and specific characteristics of navigation route
US8298231B2 (en) * 2008-01-31 2012-10-30 Tyco Healthcare Group Lp Bipolar scissors for adenoid and tonsil removal
US7932690B2 (en) * 2008-02-11 2011-04-26 Robert Dion Jones Wall saver
US8540474B2 (en) 2008-07-29 2013-09-24 Thomas F. Egan Power lift and transfer system and method
US8280561B2 (en) * 2008-09-09 2012-10-02 University Health Network Powered wheelchair
JP2010118008A (en) * 2008-11-14 2010-05-27 Honda Motor Co Ltd Monitoring system for low-speed passenger mobility vehicle and the like
US8231553B2 (en) 2009-01-13 2012-07-31 Tyco Healthcare Group Lp Method for wireless control of electrosurgery
US8235917B2 (en) * 2009-01-13 2012-08-07 Tyco Healthcare Group Lp Wireless electrosurgical controller
SE533876C2 (en) 2009-05-05 2011-02-15 Control and control system for a mobile disability aid
US8810407B1 (en) * 2010-05-27 2014-08-19 Guardian Angel Navigational Concepts IP LLC Walker with illumination, location, positioning, tactile and/or sensor capabilities
US9393885B2 (en) 2010-08-05 2016-07-19 Thomas F. Egan Compact multi-motion lifting and transferring apparatus and method of operating same
US10028869B2 (en) 2010-08-05 2018-07-24 Thomas F. Egan Power lift system and method
US8959681B2 (en) 2010-12-20 2015-02-24 Hill-Rom Services, Inc. Ground sensor control of foot section retraction
US10232792B2 (en) 2011-02-03 2019-03-19 Thomas F. Egan Lifting and transferring apparatus and method
EP3435189B1 (en) 2011-04-11 2022-02-09 Crown Equipment Corporation Apparatus for efficient scheduling for multiple automated non-holonomic vehicles using a coordinated path planner
US8655588B2 (en) 2011-05-26 2014-02-18 Crown Equipment Limited Method and apparatus for providing accurate localization for an industrial vehicle
ES2901153T3 (en) 2011-07-06 2022-03-21 Max Mobility Llc Motion Based Power Drive Assist System for Wheelchairs
US20140058634A1 (en) 2012-08-24 2014-02-27 Crown Equipment Limited Method and apparatus for using unique landmarks to locate industrial vehicles at start-up
US9056754B2 (en) 2011-09-07 2015-06-16 Crown Equipment Limited Method and apparatus for using pre-positioned objects to localize an industrial vehicle
US8700250B1 (en) 2012-03-15 2014-04-15 The Boeing Company Airport transportation system
CN103364000B (en) * 2012-03-26 2016-01-27 联想(北京)有限公司 A kind of localization method and electronic equipment
GB2506208A (en) * 2012-09-25 2014-03-26 Penny & Giles Controls Ltd Control module for electric wheelchair
US8886383B2 (en) 2012-09-28 2014-11-11 Elwha Llc Automated systems, devices, and methods for transporting and supporting patients
US20140094997A1 (en) * 2012-09-28 2014-04-03 Elwha Llc Automated Systems, Devices, and Methods for Transporting and Supporting Patients Including Multi-Floor Operation
GB201220468D0 (en) 2012-11-14 2012-12-26 Provost Fellows Foundation Scholars And The Other Members Of Board Of A control interface
US9217535B1 (en) 2013-02-21 2015-12-22 Thomas F. Egan Portable lifting and transferring techniques
US9144525B2 (en) 2013-03-14 2015-09-29 Max Mobility, Llc. Motion assistance system for wheelchairs
TWI542339B (en) * 2013-12-12 2016-07-21 dun-ji Li Wheelchair equipment and systems for self care
US9383751B2 (en) 2013-12-12 2016-07-05 Medicraft Holdings (Taiwan) Co., Ltd. Self operable wheelchair
US20160049063A1 (en) * 2014-08-14 2016-02-18 Pauline Dennis Mobility Device Alert
US9314928B2 (en) * 2014-08-21 2016-04-19 Elwha Llc Systems, devices, and methods including a wheelchair-assist robot
US9314929B2 (en) 2014-08-21 2016-04-19 Elwha Llc Systems, devices, and methods including a wheelchair-assist robot
US10099379B2 (en) 2014-08-21 2018-10-16 Elwha Llc Systems, devices, and methods including a wheelchair-assist robot
US20160074262A1 (en) * 2014-12-26 2016-03-17 Sophia Vasiliki Moses Intelliwalker, an intelligent, sensor equipped, motorized robotic walking assistance device.
KR101657128B1 (en) * 2015-01-23 2016-09-13 한인석 An electric wheelchair having a sound recognition driving system
US9795524B2 (en) 2015-02-24 2017-10-24 Max Mobility, Llc Assistive driving system for a wheelchair
ITUB20153680A1 (en) * 2015-09-16 2017-03-16 Liquidweb Srl Assistive technology control system and related method
CN105270525B (en) * 2015-09-28 2018-02-02 小米科技有限责任公司 The control method and device of two-wheeled balance car
CN105287171B (en) * 2015-11-13 2017-12-08 深圳前海达闼云端智能科技有限公司 Blind guiding method, device and equipment
US10052246B2 (en) * 2016-03-15 2018-08-21 Denso International America, Inc. Autonomous wheelchair
CN105810007B (en) * 2016-03-29 2020-07-03 北京小米移动软件有限公司 Balance car parking method and device
JP6916812B2 (en) * 2016-04-14 2021-08-11 デカ・プロダクツ・リミテッド・パートナーシップ User control device for transporter
CN105892691A (en) * 2016-06-07 2016-08-24 京东方科技集团股份有限公司 Method and device for controlling travel tool and travel tool system
JP6278169B1 (en) * 2016-06-24 2018-02-14 パナソニック株式会社 Electric wheelchair
US10285882B2 (en) * 2017-01-16 2019-05-14 Kwok Tung Chun Wirelessly controlled wheelchair assembly with toilet accessibility
US10852730B2 (en) * 2017-02-08 2020-12-01 Brain Corporation Systems and methods for robotic mobile platforms
US10167051B1 (en) 2017-12-12 2019-01-01 Max Mobility, Llc Assistive driving system for a wheelchair and method for controlling assistive driving system
WO2019127353A1 (en) * 2017-12-29 2019-07-04 四川金瑞麒智能科学技术有限公司 Charging navigation method and system for electric wheelchair
CN108337647B (en) * 2018-01-03 2021-03-23 仙童智行(北京)科技有限公司 Wheelchair cluster management method and device and computer readable storage medium
CA3095222A1 (en) * 2018-03-27 2020-09-25 Avidbots Corp Safety systems for semi-autonomous devices and methods of using the same
IT201800007697A1 (en) * 2018-07-31 2020-01-31 Italdesign-Giugiaro SPA Transport system for the mobility of disabled people on wheelchairs.
CN117234115B (en) * 2023-11-15 2024-02-02 深圳安培时代数字能源科技有限公司 Control method and related device of electric wheelchair

Citations (27)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4278142A (en) * 1978-05-08 1981-07-14 Agency Of Industrial Science And Technology Automatic guidance system for vehicles
US4626993A (en) * 1982-02-19 1986-12-02 Kubota, Ltd. Teaching-playback type automatically running vehicle
US4647784A (en) * 1983-05-14 1987-03-03 The General Electric Company Plc Vehicle guidance and control system
US4817000A (en) * 1986-03-10 1989-03-28 Si Handling Systems, Inc. Automatic guided vehicle system
US5006988A (en) * 1989-04-28 1991-04-09 University Of Michigan Obstacle-avoiding navigation system
US5033000A (en) * 1988-06-09 1991-07-16 Natco Corporation Variable keyed power distribution and control system for motorized wheelchair
US5051906A (en) * 1989-06-07 1991-09-24 Transitions Research Corporation Mobile robot navigation employing retroreflective ceiling features
US5076384A (en) * 1989-10-17 1991-12-31 Mitsubishi Denki Kabushiki Kaisha Ultrasonic obstacle sensor
US5155684A (en) * 1988-10-25 1992-10-13 Tennant Company Guiding an unmanned vehicle by reference to overhead features
US5363933A (en) * 1992-08-20 1994-11-15 Industrial Technology Research Institute Automated carrier
US5497056A (en) * 1994-05-10 1996-03-05 Trenton State College Method and system for controlling a motorized wheelchair using controlled braking and incremental discrete speeds
US5545960A (en) * 1991-04-09 1996-08-13 International Business Machines Corporation Autonomous mobile machine, and system and method for controlling a mobile machine
US5677836A (en) * 1994-03-11 1997-10-14 Siemens Aktiengesellschaft Method for producing a cellularly structured environment map of a self-propelled, mobile unit that orients itself in the environment at least with the assistance of sensors based on wave refection
US5687136A (en) * 1996-04-04 1997-11-11 The Regents Of The University Of Michigan User-driven active guidance system
US5793934A (en) * 1994-06-22 1998-08-11 Siemens Aktiengesellschaft Method for the orientation, route planning and control of an autonomous mobile unit
US5804942A (en) * 1995-08-08 1998-09-08 Samsung Electronics Co., Ltd. Position determining apparatus and control method of robot
US5812978A (en) * 1996-12-09 1998-09-22 Tracer Round Associaties, Ltd. Wheelchair voice control apparatus
US5911767A (en) * 1994-10-04 1999-06-15 Garibotto; Giovanni Navigation system for an autonomous mobile robot
US5961571A (en) * 1994-12-27 1999-10-05 Siemens Corporated Research, Inc Method and apparatus for automatically tracking the location of vehicles
US5999866A (en) * 1996-11-05 1999-12-07 Carnegie Mellon University Infrastructure independent position determining system
US6078849A (en) * 1998-12-11 2000-06-20 Lockheed Martin Corporation Vision based precision docking of vehicles
US6108592A (en) * 1998-05-07 2000-08-22 International Business Machines Corporation Voice-controlled motorized wheelchair with sensors and displays
US6125191A (en) * 1995-05-22 2000-09-26 Honda Giken Kogyo Kabushiki Kaisha Object detecting system for vehicle
US6135228A (en) * 1996-04-25 2000-10-24 Massachusetts Institute Of Technology Human transport system with dead reckoning facilitating docking
US6154690A (en) * 1999-10-08 2000-11-28 Coleman; Raquel Multi-feature automated wheelchair
US6195610B1 (en) * 1996-01-18 2001-02-27 Komatsu Ltd. Teaching method and apparatus of unmanned vehicle traveling course
US6205380B1 (en) * 1996-07-02 2001-03-20 Siemens Aktiengesellschaft Process for preparing an area plan having a cellular structure and comprising a unit moving automatically and positioned in said area using sensors based on wave reflection

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5961561A (en) * 1997-08-14 1999-10-05 Invacare Corporation Method and apparatus for remote maintenance, troubleshooting, and repair of a motorized wheelchair

Patent Citations (27)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4278142A (en) * 1978-05-08 1981-07-14 Agency Of Industrial Science And Technology Automatic guidance system for vehicles
US4626993A (en) * 1982-02-19 1986-12-02 Kubota, Ltd. Teaching-playback type automatically running vehicle
US4647784A (en) * 1983-05-14 1987-03-03 The General Electric Company Plc Vehicle guidance and control system
US4817000A (en) * 1986-03-10 1989-03-28 Si Handling Systems, Inc. Automatic guided vehicle system
US5033000A (en) * 1988-06-09 1991-07-16 Natco Corporation Variable keyed power distribution and control system for motorized wheelchair
US5155684A (en) * 1988-10-25 1992-10-13 Tennant Company Guiding an unmanned vehicle by reference to overhead features
US5006988A (en) * 1989-04-28 1991-04-09 University Of Michigan Obstacle-avoiding navigation system
US5051906A (en) * 1989-06-07 1991-09-24 Transitions Research Corporation Mobile robot navigation employing retroreflective ceiling features
US5076384A (en) * 1989-10-17 1991-12-31 Mitsubishi Denki Kabushiki Kaisha Ultrasonic obstacle sensor
US5545960A (en) * 1991-04-09 1996-08-13 International Business Machines Corporation Autonomous mobile machine, and system and method for controlling a mobile machine
US5363933A (en) * 1992-08-20 1994-11-15 Industrial Technology Research Institute Automated carrier
US5677836A (en) * 1994-03-11 1997-10-14 Siemens Aktiengesellschaft Method for producing a cellularly structured environment map of a self-propelled, mobile unit that orients itself in the environment at least with the assistance of sensors based on wave refection
US5497056A (en) * 1994-05-10 1996-03-05 Trenton State College Method and system for controlling a motorized wheelchair using controlled braking and incremental discrete speeds
US5793934A (en) * 1994-06-22 1998-08-11 Siemens Aktiengesellschaft Method for the orientation, route planning and control of an autonomous mobile unit
US5911767A (en) * 1994-10-04 1999-06-15 Garibotto; Giovanni Navigation system for an autonomous mobile robot
US5961571A (en) * 1994-12-27 1999-10-05 Siemens Corporated Research, Inc Method and apparatus for automatically tracking the location of vehicles
US6125191A (en) * 1995-05-22 2000-09-26 Honda Giken Kogyo Kabushiki Kaisha Object detecting system for vehicle
US5804942A (en) * 1995-08-08 1998-09-08 Samsung Electronics Co., Ltd. Position determining apparatus and control method of robot
US6195610B1 (en) * 1996-01-18 2001-02-27 Komatsu Ltd. Teaching method and apparatus of unmanned vehicle traveling course
US5687136A (en) * 1996-04-04 1997-11-11 The Regents Of The University Of Michigan User-driven active guidance system
US6135228A (en) * 1996-04-25 2000-10-24 Massachusetts Institute Of Technology Human transport system with dead reckoning facilitating docking
US6205380B1 (en) * 1996-07-02 2001-03-20 Siemens Aktiengesellschaft Process for preparing an area plan having a cellular structure and comprising a unit moving automatically and positioned in said area using sensors based on wave reflection
US5999866A (en) * 1996-11-05 1999-12-07 Carnegie Mellon University Infrastructure independent position determining system
US5812978A (en) * 1996-12-09 1998-09-22 Tracer Round Associaties, Ltd. Wheelchair voice control apparatus
US6108592A (en) * 1998-05-07 2000-08-22 International Business Machines Corporation Voice-controlled motorized wheelchair with sensors and displays
US6078849A (en) * 1998-12-11 2000-06-20 Lockheed Martin Corporation Vision based precision docking of vehicles
US6154690A (en) * 1999-10-08 2000-11-28 Coleman; Raquel Multi-feature automated wheelchair

Cited By (43)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7594556B1 (en) * 2004-08-27 2009-09-29 Cook Technologies, Inc. System for storing and retrieving a personal-transportation vehicle
US8312953B1 (en) * 2004-08-27 2012-11-20 Fs Partners, Llc System for storing and retrieving a personal-transportation vehicle
WO2009046282A1 (en) * 2007-10-03 2009-04-09 University Of Delaware Et Al. Intelligent powered mobility for infants and special needs children
US20090132124A1 (en) * 2007-10-03 2009-05-21 University Of Delaware Intelligent powered mobility for infants and special needs children
US8090488B2 (en) 2007-10-03 2012-01-03 Agrawal Sunil K Intelligent powered mobility for infants and special needs children
US20100123574A1 (en) * 2008-11-14 2010-05-20 Honda Motor Co., Ltd. Monitoring system for vehicle
US8749374B2 (en) * 2008-11-14 2014-06-10 Honda Motor Co., Ltd. Monitoring system for vehicle
US8731738B2 (en) 2010-08-31 2014-05-20 University Of Delaware Powered mobility systems and methods
US20120136666A1 (en) * 2010-11-29 2012-05-31 Corpier Greg L Automated personal assistance system
US8924218B2 (en) * 2010-11-29 2014-12-30 Greg L. Corpier Automated personal assistance system
WO2014011992A2 (en) * 2012-07-13 2014-01-16 Love Park Robotics, Llc Drive-control systems for vehicles such as personal-transportation vehicles
WO2014011992A3 (en) * 2012-07-13 2014-03-27 Love Park Robotics, Llc Drive-control systems for vehicles such as personal-transportation vehicles
US9757054B2 (en) 2013-08-30 2017-09-12 Elwha Llc Systems and methods for warning of a protruding body part of a wheelchair occupant
US10271772B2 (en) 2013-08-30 2019-04-30 Elwha Llc Systems and methods for warning of a protruding body part of a wheelchair occupant
US9488482B2 (en) 2013-08-30 2016-11-08 Elwha Llc Systems and methods for adjusting a contour of a vehicle based on a protrusion
US10030991B2 (en) 2013-08-30 2018-07-24 Elwha Llc Systems and methods for adjusting a contour of a vehicle based on a protrusion
US9915545B2 (en) 2014-01-14 2018-03-13 Toyota Motor Engineering & Manufacturing North America, Inc. Smart necklace with stereo vision and onboard processing
US10360907B2 (en) 2014-01-14 2019-07-23 Toyota Motor Engineering & Manufacturing North America, Inc. Smart necklace with stereo vision and onboard processing
US9629774B2 (en) 2014-01-14 2017-04-25 Toyota Motor Engineering & Manufacturing North America, Inc. Smart necklace with stereo vision and onboard processing
US9578307B2 (en) 2014-01-14 2017-02-21 Toyota Motor Engineering & Manufacturing North America, Inc. Smart necklace with stereo vision and onboard processing
US10024679B2 (en) 2014-01-14 2018-07-17 Toyota Motor Engineering & Manufacturing North America, Inc. Smart necklace with stereo vision and onboard processing
US10248856B2 (en) 2014-01-14 2019-04-02 Toyota Motor Engineering & Manufacturing North America, Inc. Smart necklace with stereo vision and onboard processing
US9919694B1 (en) 2014-04-28 2018-03-20 Ball Aerospace & Technologies Corp. Control system for a complex mechanical system
US9316502B2 (en) * 2014-07-22 2016-04-19 Toyota Motor Engineering & Manufacturing North America, Inc. Intelligent mobility aid device and method of navigating and providing assistance to a user thereof
US10024667B2 (en) 2014-08-01 2018-07-17 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable earpiece for providing social and environmental awareness
US10024678B2 (en) 2014-09-17 2018-07-17 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable clip for providing social and environmental awareness
US9922236B2 (en) 2014-09-17 2018-03-20 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable eyeglasses for providing social and environmental awareness
USD768024S1 (en) 2014-09-22 2016-10-04 Toyota Motor Engineering & Manufacturing North America, Inc. Necklace with a built in guidance device
US9576460B2 (en) 2015-01-21 2017-02-21 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable smart device for hazard detection and warning based on image and audio data
US10490102B2 (en) 2015-02-10 2019-11-26 Toyota Motor Engineering & Manufacturing North America, Inc. System and method for braille assistance
US9586318B2 (en) 2015-02-27 2017-03-07 Toyota Motor Engineering & Manufacturing North America, Inc. Modular robot with smart device
US10391631B2 (en) 2015-02-27 2019-08-27 Toyota Motor Engineering & Manufacturing North America, Inc. Modular robot with smart device
US9811752B2 (en) 2015-03-10 2017-11-07 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable smart device and method for redundant object identification
US9677901B2 (en) 2015-03-10 2017-06-13 Toyota Motor Engineering & Manufacturing North America, Inc. System and method for providing navigation instructions at optimal times
US9972216B2 (en) 2015-03-20 2018-05-15 Toyota Motor Engineering & Manufacturing North America, Inc. System and method for storing and playback of information for blind users
US9898039B2 (en) 2015-08-03 2018-02-20 Toyota Motor Engineering & Manufacturing North America, Inc. Modular smart necklace
US10024680B2 (en) 2016-03-11 2018-07-17 Toyota Motor Engineering & Manufacturing North America, Inc. Step based guidance system
US9958275B2 (en) 2016-05-31 2018-05-01 Toyota Motor Engineering & Manufacturing North America, Inc. System and method for wearable smart device communications
US10561519B2 (en) 2016-07-20 2020-02-18 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable computing device having a curved back to reduce pressure on vertebrae
US10432851B2 (en) 2016-10-28 2019-10-01 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable computing device for detecting photography
US10012505B2 (en) 2016-11-11 2018-07-03 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable system for providing walking directions
US10521669B2 (en) 2016-11-14 2019-12-31 Toyota Motor Engineering & Manufacturing North America, Inc. System and method for providing guidance or feedback to a user
US10172760B2 (en) 2017-01-19 2019-01-08 Jennifer Hendrix Responsive route guidance and identification system

Also Published As

Publication number Publication date
AU2003281326A8 (en) 2004-01-23
US20040006422A1 (en) 2004-01-08
US6842692B2 (en) 2005-01-11
WO2004005852A1 (en) 2004-01-15
AU2003281326A1 (en) 2004-01-23
US7383107B2 (en) 2008-06-03

Similar Documents

Publication Publication Date Title
US7383107B2 (en) Computer-controlled power wheelchair navigation system
US20080300777A1 (en) Computer-controlled power wheelchair navigation system
Yanco Wheelesley: A robotic wheelchair system: Indoor navigation and user interface
JP7395229B2 (en) Mobile cleaning robot artificial intelligence for situational awareness
Simpson Smart wheelchairs: A literature review.
Madarasz et al. The design of an autonomous vehicle for the disabled
Miller et al. Design and testing of a low-cost robotic wheelchair prototype
Zeng et al. A collaborative wheelchair system
Kulyukin et al. Robot-assisted wayfinding for the visually impaired in structured indoor environments
Park et al. Robotic smart house to assist people with movement disabilities
Vanhooydonck et al. Shared control for intelligent wheelchairs: an implicit estimation of the user intention
Sekmen et al. An application of passive human-robot interaction: human tracking based on attention distraction
Chaccour et al. Computer vision guidance system for indoor navigation of visually impaired people
Crisman et al. Progress on the deictically controlled wheelchair
JP2004042148A (en) Mobile robot
WO2022247325A1 (en) Navigation method for walking-aid robot, and walking-aid robot and computer-readable storage medium
Chaccour et al. Novel indoor navigation system for visually impaired and blind people
Demeester et al. A model-based, probabilistic framework for plan recognition in shared wheelchair control: Experiments and evaluation
Ghidary et al. Multi-modal interaction of human and home robot in the context of room map generation
Jayakody et al. Smart wheelchair to facilitate disabled individuals
US20230111676A1 (en) Mobile robotic arm configured to provide on-demand assistance
Trahanias et al. Navigational support for robotic wheelchair platforms: an approach that combines vision and range sensors
Andersson et al. ISR: An intelligent service robot
Ghidary et al. Multi-modal human robot interaction for map generation
JP5115886B2 (en) Road guidance robot

Legal Events

Date Code Title Description
AS Assignment

Owner name: U.S. DEPARTMENT OF VETERANS AFFAIRS OF EDWARD HINE

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:FEHR, LINDA;SKAAR, STEVEN B.;DEL CASTILLO, GUILLERMO;REEL/FRAME:015578/0733;SIGNING DATES FROM 20040628 TO 20040703

STCF Information on status: patent grant

Free format text: PATENTED CASE

FPAY Fee payment

Year of fee payment: 4

REMI Maintenance fee reminder mailed
FPAY Fee payment

Year of fee payment: 8

SULP Surcharge for late payment

Year of fee payment: 7

FEPP Fee payment procedure

Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

LAPS Lapse for failure to pay maintenance fees

Free format text: PATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

STCH Information on status: patent discontinuation

Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362

FP Lapsed due to failure to pay maintenance fee

Effective date: 20200603