WO2009098319A2

WO2009098319A2 - Navigational device for a vehicle

Info

Publication number: WO2009098319A2
Application number: PCT/EP2009/051455
Authority: WO
Inventors: Mark Richard Tucker
Original assignee: Trw Limited
Priority date: 2008-02-09
Filing date: 2009-02-09
Publication date: 2009-08-13
Also published as: WO2009098319A3; GB0802444D0

Abstract

A navigation system comprising a navigational device suitable for fitment on or carriage by a vehicle (100) and at least one positional marker (400) located in a fixed position relative to a highway along which the vehicle can travel. The navigational device includes a processing means arranged to receive a feed from a camera including an image of a scene ahead of the vehicle (100), process the image to identify any positional markers (400) in the scene, determine the absolute position of the positional marker (400) using information associated with the marker, determine the distance between the vehicle (100) and the positional marker (400) by processing of the image fed from the camera; and from the absolute position and distance determine an estimate of the position of the vehicle (100).

Description

NAVIGATIONAL DEVICE FOR A VEHICLE

This invention relates to navigational devices and to a vehicle incorporating a navigational device. It may also relate to a vehicle location determining apparatus that can be used to control one or more operational parameters of a vehicle as a function of vehicle position. It is especially suited to navigational devices for road vehicles such as cars and trucks, but is not limited to this. Other vehicles such as autonomous vehicles in factories or vehicles in automated transport systems may utilise this invention.

Navigational devices based on GPS are well known. They are widely fitted to vehicles to provide direction and mapping functions to the driver or passenger. In their most basic form, these devices comprise a processing device that receives a signal indicative of an absolute position on the earth fed from a GPS receiver. The receiver determines its position by measuring the distance between itself and three or more GPS satellites using trilateration. This position signal can be used to look-up information in a database, such as a region of map centred on that position. This information can then be displayed on a screen. In addition, many of these devices are able to plot routes between two points, and to display the route on the display, often overlayed on a stylised map.

The accuracy of commercially available GPS signals is limited to a few metres in any direction about the actual position due to many factors, such as atmospheric effects on the microwave signals emitted by the GPS satellites. With additional processing this accuracy can be improved to a few centimetres, although only under certain conditions. Quite often it is not possible to receive a signal at all, such as when passing through a tunnel or when indoors such as a covered car park or factory. At other times, the signal may be inaccurate due to reflections of the incoming signals from tall buildings in built up areas.

Where the signal is lost or inaccurate, it may be impossible to calculate or estimate a position, preventing an accurate map display on the screen.

A prior art solution to this problem is known in which dead reckoning is used following loss of a signal. The device may receive a feed from a wheel speed sensor and use this to work out how far the vehicle has moved since the signal was lost. If this is combined with steering wheel angle a reasonably accurate measure of position can be retained for some time after a loss of GPS signal.

This prior art solution is limited in several ways. In particular it is of no help where the received signal is simply inaccurate rather than lost. Also, it is complicated to pick up the wheel speed signal to give an accurate measure of distance moved. A change in rolling diameter of the wheel, perhaps due to the replacement of an original wheel and tyre combination with one of a different size will lead to errors. Wheel spin on slippery surfaces may also lead to an error in any dead reckoning technique.

In an alternative it is known for a vehicle to determine its position using markings applied to the highway or to markers alongside the highway. An example of this is the vehicles used in the United Kingdom to measure wear on the surface of the motorway network. Signs carrying barcodes are located at regular intervals along the road and are detected by the vehicle. The barcode carries position information that can be identified by an operator of the vehicle or passenger and subsequently logged. This allows the operator to confirm that they have driven past a set point on the highway. An object of the present invention is to ameliorate at least some of the problems with inaccuracies in position determined by prior art navigational devices.

According to a first aspect the invention provides a navigation system comprising: a navigational device suitable for fitment to or carriage by a vehicle and at least one positional marker located in a fixed position relative to a highway along which the vehicle can travel; in which the navigational device includes a processing means arranged to:

(a) receive a feed from a camera including an image of a scene ahead of the vehicle;

(b) process the image to identify any positional markers in the scene;

(c) determine the absolute position of the positional marker using information associated with the marker;

(d) determine the distance between the vehicle and the positional marker by processing of the image fed from the camera; and

(e) from the absolute position and distance determined in steps (c) and (d) determine an estimate of the position of the vehicle.

By using a camera to determine the position of the vehicle relative to a marker fixed by a highway, and also determining the absolute position of that marker, it is possible to determine vehicle position without relying on a GPS signal. The absolute position suggested by the marker can be "corrected" to take into account how far the vehicle is away from the marker.

The navigational device may, additionally, receive a signal from a GPS receiver indicating a position of the vehicle. This may be used in combination with any position determined from a marker identified in the camera image to give a position measurement. This could be an average taken from the GPS receiver and the camera based position measurements. Alternatively, and most preferably, the device may use whichever of the two position estimates it considers the most reliable at any time. Factors such as distance to a marker, vehicle speed and GPS signal strength may be taken into account when deciding which is most reliable at any time.

The system may include many tens or hundreds of markers. Each marker may carry information indicating its position, or information from which its position may be determined. In the later case, the information may uniquely identify the marker enabling its position to be determined by cross-reference with information relating to the marker stored in a memory.

Examples of the form in which information that may be carried by the marker include its shape, pattern on the marker, textual, numerical or alphanumerical characters, and colour. A barcode could be used, the numerical value represented by the barcode encoding the position.

The position information could in this way be directly carried by the marker, allowing the position to be determined without the use of any other additional information. In an alternative, the marker may carry information that identifies the marker, the position being later determined by looking up stored position information that corresponds to the identified marker. The information may therefore be carried directly or may be carried indirectly by the marker.

At least one of the markers may be located at a point along a highway at which it is known that a GPS signal is not available. For example, a marker may be located in a tunnel or in an area shielded by tall trees or buildings. The processing means of the navigational device may process the image by the steps of:

(i) determine the height of the marker above a highway using information associated with the marker;

(ii) identify the location of the marker in the image; (iii) identify the relative location of the highway directly below the marker in the image;

(iv) determine the spacing in the image between the marker and highway positions; and

(v) determine the distance between the camera and marker that would result in the relative spacing between the camera and marker in the image.

This process exploits the fact that the further away a marker and highway is within an image, the smaller the distance in the image for a given distance between the marker and highway.

The navigational device may determine the height of the marker by extracting information carried by the marker, either directly or indirectly, or a support for the marker, captured in the image. For example, the colour of the marker may indicate that it is a certain height. Or the support may carry stripes that are at a known spaced interval, the number of stripe indicating the height. The height may simply be written on the marker. Alternatively, the height information may be combined with the position information. A barcode is an example of a suitable information carrier for this purpose and may be used within the scope of the invention.

In an alternative, the height of the marker may be stored in a memory of the device and the step of determining the height may comprise identifying the marker from the image and looking up the associated height from the memory.

The device may determine the distance between the camera and the marker by applying a perspective transform to the image captured by the camera. This may take into account the height of the camera above the highway and its orientation relative to the vehicle. Because the transform uses the height of the marker above the highway, the actual portion of the image used in the transform is much greater than it would be if simply relying on knowledge of the size of the marker alone.

The navigational device may include a display screen, and may display a map of a region around the determined position on the display screen.

The processing means may comprise an image processor and a data processor connected to one or more areas of electronic memory in which are stored program instructions. The instructions may include one or more routines and sub-routines that cause the processing means to perform each of the claimed functional steps.

The image processor and processor may be combined as a single processing device.

The marker may be identified in a captured image using pattern matching techniques known in the art, whereby portions of the image are compared with a known pattern that corresponds to the marker. For example, where a marker is identified by shape and carries a barcode containing position information, the image may be searched to identify the marker shape, and the area of image within the shape searched to identify the barcode. The device may include a compass. This may be used to decide the direction in which the correction to the position indicated by the marker is to be made. For example, if it is determined that the marker is 100m away and to the south, the latitude of the car can be estimated as being 100m out to the north with no longitude correction needed.

In an alternative to providing a compass the heading angle of the vehicle may be determined from an available GPS signal either at the time of determining position or some time previous to that.

The position determined by the system can be expressed in a number of different ways. A preferred format for a wide area system such as a national network of highways is to express the system in terms of longitude and latitude relative to the earth's surface. For a system that is to navigate in a smaller, confined area such as a factory floor, a proprietary co-ordinate system may be used instead.

The system may include a camera fitted to the vehicle which may capture an image of the highway ahead of the vehicle. The camera may comprise a CCD camera.

According to a second aspect the invention provides a navigational device for use in the navigational system of the first aspect.

According to a third aspect the invention provides a vehicle that includes a navigational device according to the second aspect.

According to a fourth aspect the invention provides a marker for use in the navigational system of the first aspect of the invention. The marker may carry information indicating its absolute position, or information from which its position may be determined. In the later case, the information may uniquely identify the marker enabling its position to be determined by cross-reference with information relating to the marker stored in a memory.

Examples of the form in which information that may be carried by the marker include its shape, pattern on the marker, textual, numerical or alphanumerical characters and colour.

At least one of the markers may be located at a point along a highway at which it is known that a GPS signal is not available. For example, a marker may be located in a tunnel or in an area shielded by tall trees or buildings.

Each marker may be located at the same height above the road surface, or may carry indicia from which its height can be determined from information contained in an image captured by a camera.

Whilst the invention has been described in terms of a navigation system and device, it applies equally to a system in which the position information can be used for other purposes than giving navigational instructions to a person or device. For example, the position information may be used

Therefore according to a fifth aspect the invention provides apparatus for use in combination with one or more positional markers that is capable of determining its position comprising: a processing means arranged to:

(b) process the image to identify any markers in the scene; (c) determine the position of the marker using information associated with the marker;

(d) determine the distance between the vehicle and the marker; and

(e) from the results obtained in steps (c) and (d) determines the position of the vehicle

The position information could be used, for instance, to assist in the control of an autonomous vehicle. It may be used to control one or more operational parameters of a vehicle that do not relate to navigation such as speed.

For the avoidance of doubt, any of the features described in relation to the first through fourth aspects of the invention, may be combined with the features of the fifth aspect and may form the basis of a claim.

There will now be described by way of example only one embodiment of the present invention with reference to the accompanying drawings of which:

Figure 1 is a representation of the component parts of an embodiment of a navigation device in accordance with one aspect of the present invention fitted to a vehicle;

Figure 2 is a schematic of the key functional parts of the processing unit of the device of Figure 1 ;

Figure 3 is an illustration of a navigation system that includes the vehicle with the navigation device of Figure 1 and a set of markers located around a network of highways;

Figure 4 is a closer view of one of the markers of Figure 3; and Figure 5 is a sample image captured by the camera of the device of Figure 1 showing a marker in the scene captured in the image.

Figure 1 illustrates the component parts of a navigation device that can be fitted to a vehicle 100. The device is distributed around the vehicle, and its key components are a forward facing camera 210 located perhaps behind the front grille, a processing unit 220 that may be located within the passenger compartment behind the dashboard and a display 230 located within the dashboard so that it easily is viewed by a driver. The device also includes an optional GPS receiver 240 that receives electrical signals from an antenna located on the roof of the vehicle 100. This location gives the antenna a clear line of sight up into the sky towards the satellites that provide the basis of the GPS signal. Although the illustration shows that the components are physically connected through dedicated wiring, they could all be connected to a bus network on the vehicle such as a CAN bus that allows them to exchange information as required.

The device could, however, be formed with all of these parts combined in one housing, possibly a small hand held portable housing. Such a housing could be located above the dashboard inside the windscreen. This position would give a forward view to the camera and an upward view to the sky.

The camera 210 in the embodiment is in fact a camera that forms a part of a lane guidance system (not shown) . A feed from the camera 210 is fed to the lane guidance system and a tap from the feed is passed to the processing unit 220 of the navigation device. This feed is passed to an image processor forming part of the processing unit 220 that converts the signal into captured image frames that can subsequently be analysed. Reusing an existing camera in this way to provide part of the navigational device reduces its complexity and overall cost to the end user.

The key elements of the processing unit 220 are shown in more detail in the schematic diagram of figure 2 of the accompanying drawings. It comprises a circuit board 201 that carries a processor 202, a memory 203 connected to the processor and a dedicated image processor device 204. Power to the board and the processors is fed from the vehicle battery as is known in the art through tracks 205 on the circuit board.

The memory 203 stores a set of program instructions that are executed by the processor of the navigational device whenever power is fed to the device. The power may be connected permanently - a live feed, or whenever the vehicle is switched on- an ignition switched feed. To provide optimum conservation of the vehicle battery power and flexibility, in the example a switched live feed is used which can be turned on or off by the driver independent of operation of the ignition.

The processor 201 , when executing the stored instructions, initially looks for a position signal from the GPS receiver 204. The receiver is always on when the device is powered up, and so in normal operating conditions provides a continuously updated output signal containing position information. Of course, if the optional GPS receiver is not present this step will be omitted.

Unfortunately, it is not unknown that a GPS receiver may lose sight of the satellites that it derives its position signal from, or the signals from them may become too weak to be used reliably. In the event of loss of a GPS signal, the navigational device can continue to determine position using information derived from the images captured by the camera 210. To assist this function, a set of markers is provided around the highway network. They can be located at any point, but are ideally located at points where a decision has to be made by a driver, such as a junction. They can also be used to advantage by placing them at points where GPS signal strength is weak. An overview of a set of markers 310 distributed around a road network 300 is shown in Figure 3 of the accompanying drawings. The markers 310 are fitted to traffic lights, and also located under a bridge or tunnel 320, between tall buildings 330 and under heavy tree cover 340. These are all areas in which a reliable GPS signal is unlikely to be received. The markers and navigation device form a system within the scope of the first aspect of the invention. It is the identification of these markers 310 in the captured images, and subsequent analysis of the relative position of these markers and the vehicle that gives the more robust performance compared with use of GPS only.

Figure 4 illustrates a sample marker (one of a set) located at a position along a highway. By this we mean that it is located so that it will appear in the field of view of the camera fitted to the vehicle 100 as it moves towards the marker along the highway. Suitable fixing locations will include overhead gantries, roadside support poles, traffic light supports or perhaps the side of a building or wall. The marker carries a barcode that represents a unique coded value containing information that identifies the marker. In fact, in this example the coded value also indicates the exact position of the marker (longitude and latitude) and how high the marker is located above the highway surface. This is not always essential provided that the navigation device has some other way of determining the position and/or height from the identity of the marker. It could perform this task using a database stored in memory, or by interrogating a remotely stored database that cross references marker identity with position and height information.

In a first step, the processing unit looks to the output of the image processor to see if there are any images being captured from the camera. A typical image captured by the camera is shown in Figure 5 of the accompanying drawings. Here a marker 400 can be identified alongside a stretch of highway.

In a next step, the image is analysed to determine whether or not a marker is present in the image (in the example it is present) . As will become apparent, the provision of markers at set points around a highway provides increased robustness to the position determined by the navigational device.

Where a marker is identified, the next step performed by the processing unit is to identify the position of the marker. In this example, this is achieved by reading or "scanning" the barcode in the image that is provided on the marker.

Once the position of the marker is identified as well as its height, the actual height is used to work out the distance between the camera and the marker that would produce the image of the marker and highway as seen in the captured image. This exploits the fact that the image will appear different if the camera is closer to the marker than it would when further away. In particular, the spacing in the image between the marker and the highway surface next to or below it will be smaller when viewed from afar than when viewed at close quarters.

To determine the distance between camera and marker in this way, the processor in this embodiment first determines the location of the road surface/ ground plane in the image captured by the camera at the location of the marker and then applies a perspective transform to the image about this ground plane.

Once the distance to the marker is known, it is used to determine the actual position of the vehicle. For example, if it is known that the marker is 100m ahead of the vehicle, then the actual position of the vehicle will be given by the position of the marker corrected by 100m. A compass fitted to the vehicle can be used to enable the device to decide in which direction the 100m correction should be made

This position information is then used to replace any previous position information and stored in the memory.

The process is repeated continuously either on each capture of an image from the camera or at a lower rate. It may be simpler to process the position information every second, rather than on each frame. This reduces the processing speed and power needed with little loss of accuracy as a vehicle cannot travel far in that time.

Where GPS is provided, as in this embodiment, the image processing may cease when a GPS signal is regained.

It is possible to make various modifications to the described embodiment without departing from the claimed scope of the invention. The GPS receiver could be omitted if enough markers are located around the highway. For example, if a vehicle can always see at least one marker at all times, it would always be possible to work out vehicle position from the captured images. Alternatively, both GPS and camera based position data may be combined to give an increase in accuracy by taking an average of the two, or using the camera based information to override the GPS signal wherever possible. This will give an increased accuracy wherever GPS has a low spatial accuracy, as is often the case at the time of writing due to deliberately introduced errors in the source of the GPS signals.

Claims

1. A navigation system comprising: a navigational device suitable for fitment on or carriage by a vehicle and at least one positional marker located in a fixed position relative to a highway along which the vehicle can travel; in which the navigational device includes a processing means arranged to: (a) receive a feed from a camera including an image of a scene ahead of the vehicle; (b) process the image to identify any positional markers in the scene;

(d) determine the distance between the vehicle and the positional marker by processing of the image fed from the camera; and (e) from the absolute position and distance determined in steps (c) and (d) determine an estimate of the position of the vehicle.

2. The navigational device of claim 1 which additionally receives a signal from a GPS receiver indicating a position of the vehicle which signal is used in combination or as an alternative to any position determined from a marker identified in the camera image to give a position measurement.

3. The navigational device of claim 1 or 2 in which the processing means processes the image by the steps of:

(i) determine the height of the marker above a highway using information associated with the marker; (ii) identify the location of the marker in the image; (iii) identify the relative location of the highway directly below the marker in the image; (iv) determine the spacing in the image between the marker and highway positions; and

4. The navigational device of claim 3 in which the processing means determines the height of the marker in step (i) by extracting information carried by the marker, either directly or indirectly, or a support for the marker, captured in the image.

5. The navigational device of claim 3 or 4 in which the processing means determines the distance between the camera and the marker in step (v) by applying a perspective transform to the image captured by the camera.

6. A navigational device for use in the navigational system of any one of claims 1 to 5.

7. A marker for use in the navigational system of any one of claims 1 to 5.

8. Apparatus for use in combination with one or more positional markers that is capable of determining its position comprising: a processing means arranged to:

(b) process the image to identify any markers in the scene;

(c) determine the position of the marker using information associated with the marker; (d) determine the distance between the vehicle and the marker by processing the image; and

(e) from the results obtained in steps (c) and (d) determines the position of the vehicle.