| Pointing device and cursor for use in intelligent computing environments -> Monitor Keywords |
|
|
  Pointing device and cursor for use in intelligent computing environmentsUSPTO Application #: 20060109245Title: Pointing device and cursor for use in intelligent computing environments Abstract: A system and process for directing a laser beam within a space is presented. The system includes a pointing device which periodically outputs orientation data indicative of the direction it is pointing and a cursor device which projects a laser beam. The orientation data is used to compute the direction the pointing device is pointing in terms of yaw and pitch angles. The laser beam is directed to locations in the space relative to the amount of movement of the pointing device. In an absolute mode, the pointed device and the laser beam of the cursor device are pointed at the same location in the space, whereas in a relative mode, the pointing device does not point at the same location as the laser beam. The position of the laser can be used to select a variety of hardware devices known to be in a room for future control actions. (end of abstract) Agent: Microsoft Corporation C/o Lyon & Harr, LLP - Oxnard, CA, US Inventors: Andrew Wilson, Hubert Pham USPTO Applicaton #: 20060109245 - Class: 345157000 (USPTO) The Patent Description & Claims data below is from USPTO Patent Application 20060109245. Brief Patent Description - Full Patent Description - Patent Application Claims
CROSS-REFERENCE TO RELATED APPLICATIONS [0001] This application is a continuation of a prior application entitled "A POINTING DEVICE AND CURSOR FOR USE IN INTELLIGENT COMPUTING ENVIRONMENTS" which was assigned Ser. No. 10/461,646 and filed Jun. 13, 2003. BACKGROUND [0002] 1. Technical Field [0003] The invention is related to cursor devices, and more particularly to a system and process for directing a laser beam within a space to act as a cursor in an intelligent computing environment. [0004] 2. Background Art [0005] Ubiquitous (i.e., intelligent) computing promises to blur the boundaries between traditional desktop computing and the everyday physical world. A popular vision of tomorrow's computing pushes computational abilities into everyday objects, each participating in a complex and powerful integrated intelligent environment. Tomorrow's home and office environments, for example, may include a variety of small and large networked displays and smart controllable devices. For instance, the modern living room typically features a television, amplifier, DVD player, lights, computers, and so on. In the near future, these devices will become more inter-connected, more numerous and more specialized as part of an increasingly complex and powerful integrated intelligent environment. [0006] This migration away from the desktop and "into the walls" presents several challenges for user interface design. For example, how does the user of tomorrow's intelligent environment select one of many devices? Today, this problem is most often addressed by maintaining a separate interface, such as an IR remote control, for each device. [0007] Tomorrow's intelligent environment presents the opportunity to present a single intelligent user interface (UI) to control many such devices when they are networked. This UI device should provide the user a natural interaction with intelligent environments. For example, people have become quite accustomed to pointing at a piece of electronic equipment that they want to control, owing to the extensive use of IR remote controls. It has become almost second nature for a person in a modern environment to point at the object he or she wants to control, even when it is not necessary. Take the small radio frequency (RF) key fobs that are used to lock and unlock most automobiles in the past few years as an example. Inevitably, a driver will point the free end of the key fob toward the car while pressing the lock or unlock button. This is done even though the driver could just have well pointed the fob away from the car, or even pressed the button while still in his or her pocket, owing to the RF nature of the device. Thus, a single UI device, which is pointed at electronic components or some extension thereof (e.g., a wall switch to control lighting in a room) to control these components, would represent an example of the aforementioned natural interaction that is desirable for such a device. [0008] There are some so-called "universal" remote controls on the market that are preprogrammed with the known control protocols of a litany of electronic components, or which are designed to learn the command protocol of an electronic component. Typically, such devices are limited to one transmission scheme, such as IR or RF, and so can control only electronic components operating on that scheme. However, it would be desirable if the electronic components themselves were passive in that they do not have to receive and process commands from the UI device directly, but would instead rely solely on control inputs from the aforementioned network. In this way, the UI device does not have to differentiate among various electronic components, say by recognizing the component in some manner and transmitting commands using some encoding scheme applicable only to that component, as is the case with existing universal remote controls. [0009] Of course, a common control protocol could be implemented such that all the controllable electronic components within an environment use the same control protocol and transmission scheme. However, this would require all the electronic components to be customized to the protocol and transmission scheme, or to be modified to recognize the protocol and scheme. This could add considerably to the cost of a "single UI-controlled" environment. It would be much more desirable if the UI device could be used to control any networked group of new or existing electronic components regardless of remote control protocols or transmission schemes the components were intended to operate under. [0010] It is noted that in the remainder of this specification, the description refers to various individual publications identified by a numeric designator contained within a pair of brackets. For example, such a reference may be identified by reciting, "reference [1]" or simply "[1]". A listing of references including the publications corresponding to each designator can be found at the end of the Detailed Description section. SUMMARY [0011] The present invention involves a system and process for directing a laser beam within a space to act as a cursor. This cursor device, which will be referred to as the WorldCursor, is analogous to the mouse and cursor used in traditional graphic user interfaces. Namely, a user may select and interact with a physical device by positioning the cursor on the device and clicking. However, the WorldCursor goes much further. It is a solution to providing a natural, expressive interface for interaction in ubiquitous computing environments, where it is often a requirement to interact with devices beyond the desktop, and often in scenarios in which the traditional mouse and keyboard are inappropriate or unavailable. For example, the WorldCursor allows the user to point at and select items within a room, much as a mouse allows a user to point at objects on a computer display. [0012] The WorldCursor device itself includes a small platform that is typically installed on the ceiling. It has two small servo motors of the kind used in radio-controlled airplanes, and a laser, such as the red lasers used in laser pointing devices currently employed to give presentations. The first of the servos is configured so as to move the laser in a manner that controls the yaw direction of the laser beam and the other servo is configured so as to move the laser in a manner that controls the pitch direction of the laser beam. Thus, under computer control, the motors may be steered to point the laser almost anywhere in the room. [0013] One embodiment of the system employs control inputs from a conventional device such as a computer mouse, trackball, gamepad, or the like to dictate the movement of the WorldCursor laser. In this embodiment the computer controlling the movement of the laser receives movement control commands generated by one of the aforementioned movement control devices which specifies the direction the laser beam is to be pointed. The computer then provides commands to the WorldCursor device that direct the laser beam to move about the space as specified by the movement control commands. [0014] In another embodiment of the WorldCursor system, a pointing device is included that periodically outputs orientation data indicative of the direction it is pointing. In addition, the WorldCursor and pointing devices are both in communication with the aforementioned computer which runs a program that receives the orientation data output by the pointing device and computes the direction the pointing device is pointing from the received orientation data in terms of yaw and pitch angles. The program then directs the laser beam generated by the WorldCursor to a particular location in the space as determined by the direction the pointing device is pointing. [0015] The present system can further be employed to implement a process for selecting an object within a space. In general, this involves a user causing the laser beam to shine on a selectable object by manipulating the pointing device and then using the device to select the object. [0016] In tested versions of the WorldCursor system, the pointing device took the form of a hardware device referred to as the XWand, which is the subject of a co-pending U.S. patent application entitled "A SYSTEM AND PROCESS FOR SELECTING OBJECTS IN A UBIQUITOUS COMPUTING ENVIRONMENT" which was filed on May 31, 2002 and issued Ser. No. 10/160,692. The XWand is a hardware device and software system which allows the user to point at and operate various objects in the room. For example, the user may point the XWand at a known light, push a button on the XWand, and the light may turn on. [0017] More particularly, the XWand device contains onboard sensors to support the computation of orientation information and gesture recognition. These sensors include a pair of accelerometers. When motionless, these accelerometers sense the acceleration due to gravity, and each can be used to sense either the pitch or roll angle of the device. Another of the sensors is a 3-axis magnetoresistive permalloy magnetometer. This senses the direction of the Earth's magnetic field in 3 dimensions, and can be used to compute the yaw angle of the device. The values from the accelerometer and magnetometer are relayed to a host computer by radio link. These values are combined to find the absolute orientation of the device with respect to the room. This orientation is updated in real time at a rate of about 50 Hz, and is accurate to a few degrees in each of yaw, pitch and roll axes. [0018] The XWand system determines which device the user is pointing at by combining the orientation and 3-D position of the XWand with a 3-D model of the room and the devices within it. Orientation of the XWand is determined as explained above from the onboard sensors, while XWand position is determined with stereo computer vision. The 3-D model of the room and devices is entered into the system by pointing with the XWand itself in a special training mode. With the orientation, position and model of the room, it is easy to determine which if any object in the world model the XWand is pointing at. Audio feedback is provided to indicate to the user that the object is known to the system and can be controlled by the XWand, but in general little feedback is necessary since the pointing is absolute in nature. [0019] The WorldCursor improves upon the XWand system by not requiring an external position sensing technology. There are a number of drawbacks to using the external computer vision system. First, the system involves installing multiple cameras in the room. Part of the installation requires a rather precise calibration of the cameras against the geometry of the room. Secondly, the acquisition of the geometric model of the room and its devices requires a further calibration phase. Thirdly, computer vision techniques rely on having a clear line of sight to the device. Although this can be alleviated somewhat by installing more cameras, this approach can be prohibitively expensive and complex to install. Finally, installation of cameras inevitably raises privacy objections. The combination of the WorldCursor with the XWand eliminates the need for the external camera setup. [0020] In a first mode of operation of the WorldCursor, the laser beam is directed in an absolute pointing mode where the location that the laser beam is pointed is substantially the same location that the pointing device (e.g., the XWand) is pointing. The process that accomplishes this absolute pointing mode involves first computing a set of offset angles for the laser made up of respective yaw and pitch angles that define the angular distance between the origin of a spherical coordinate system associated with the WorldCursor and a prescribed origin of the spherical coordinate system for the space. Likewise, a set of offset angles for the pointing device are computed that represent the respective yaw and pitch angles defining the angular distance between the origin of a spherical coordinate system associated with the pointing device and the prescribed origin of the spherical coordinate system for the space. Next, aligning pitch and yaw angles are computed that define how far the laser must be moved in order to point the laser beam at the approximately the same point in the space that the pointing device is pointing. The aligning pitch angle is defined as the sum of the offset pitch angle of the laser and the computed pitch angle of the pointing device less its offset pitch angle, and the aligning yaw angle is defined as the sum of the offset yaw angle of the laser and the computed yaw angle of the pointing device less its offset yaw angle. Once the aligning angles are computed, the laser is moved to these angles so as to point the laser beam at approximately the same location in the space that the pointing device is pointing. [0021] It is noted that the offset angles computed above are initial angle values which are typically only valid for a part of the space. If the pointing device and WorldCursor are very close to each other, the angle values would remain valid no matter where the pointing device is directed. However, this will not be the usual case because the WorldCursor will typically be mounted on the ceiling of the space. If the pointing device is pointed outside the part of the space where the initial offset values are valid, the correspondence between the location the pointing device is pointing and the location the laser beam is shining is lost. Thus, the correspondence must be maintained in order to continue operating in the absolute pointing mode. If the user desires that the laser spot appear where the pointing device is pointed, or if the user would like to have the spot at least be in the field of the view of the user as they use the pointing device, a "clutching" procedure can be employed. To clutch the WorldCursor, the user momentarily activates a switch on the pointing device (e.g., pushes its button) whose state is included in each orientation data message sent by the pointing device. When the switch is activated, the laser stops moving with the pointing device and the user then reorients the pointing device, lining it up so that it points directly at the laser spot. When ready to resume WorldCursor control, the user reactivates the switch and the laser spot beings moving again in correspondence with the pointing device. Continue reading... Full patent description for Pointing device and cursor for use in intelligent computing environments Brief Patent Description - Full Patent Description - Patent Application Claims Click on the above for other options relating to this Pointing device and cursor for use in intelligent computing environments patent application. ###  How KEYWORD MONITOR works... a FREE service from FreshPatents1. Sign up (takes 30 seconds). 2. Fill in the keywords to be monitored. 3. Each week you receive an email with patent applications related to your keywords. Start now! - Receive info on patent apps like Pointing device and cursor for use in intelligent computing environments or other areas of interest. ### Previous Patent Application: Manually operated control Next Patent Application: Battery-receiving structure of wireless mouse Industry Class: Computer graphics processing, operator interface processing, and selective visual display systems ### FreshPatents.com Support Thank you for viewing the Pointing device and cursor for use in intelligent computing environments patent info. IP-related news and info Results in 5.50877 seconds Other interesting Feshpatents.com categories: Medical: Surgery , Surgery(2) , Surgery(3) , Drug , Drug(2) , Prosthesis , Dentistry |
||
