FreshPatents.com Logo
stats FreshPatents Stats
n/a views for this patent on FreshPatents.com
Updated: December 09 2014
newTOP 200 Companies filing patents this week


Advertise Here
Promote your product, service and ideas.

    Free Services  

  • MONITOR KEYWORDS
  • Enter keywords & we'll notify you when a new patent matches your request (weekly update).

  • ORGANIZER
  • Save & organize patents so you can view them later.

  • RSS rss
  • Create custom RSS feeds. Track keywords without receiving email.

  • ARCHIVE
  • View the last few months of your Keyword emails.

  • COMPANY DIRECTORY
  • Patents sorted by company.

Your Message Here

Follow us on Twitter
twitter icon@FreshPatents

Methods and apparatus for delivering information of various types to a user

last patentdownload pdfdownload imgimage previewnext patent

20130019202 patent thumbnailZoom

Methods and apparatus for delivering information of various types to a user


Some embodiments relate to techniques for performing a search for content, in which a user may issue a search query, and the search engine or engines to which that query is provided may be determined dynamically based on any of a variety of factors. For example, in some embodiments, the search engine or engines to which the query is provided may be determined based on the content of the search query, and/or auxiliary information such as the user's location, demographics, query history and/or browsing history.
Related Terms: Graphics Search Engine Graph
Browse recent Nuance Communications, Inc. patents
USPTO Applicaton #: #20130019202 - Class: 715810 (USPTO) - 01/17/13 - Class 715 
Data Processing: Presentation Processing Of Document, Operator Interface Processing, And Screen Saver Display Processing > Operator Interface (e.g., Graphical User Interface) >On-screen Workspace Or Object >Menu Or Selectable Iconic Array (e.g., Palette)



Inventors: Marc W. Regan, Vladimir Sejnoha, Sean P. Brown

view organizer monitor keywords


The Patent Description & Claims data below is from USPTO Patent Application 20130019202, Methods and apparatus for delivering information of various types to a user.

last patentpdficondownload pdfimage previewnext patent

RELATED APPLICATION

This application is a continuation of U.S. patent application Ser. No. 13/182,512, entitled “Methods And Apparatus For Identifying And Providing Information Sought By A User,” filed Jul. 14, 2011, bearing Attorney Docket No. N0484.70993US00, which is incorporated herein by reference in its entirety.

BACKGROUND

1. Field of Invention

The techniques described herein are directed generally to the field of search queries on a computer network.

2. Description of the Related Art

The Internet is a global system of interconnected computer networks that provide access to a vast array of information. The World Wide Web (WWW) is an information sharing model built on top of the Internet, in which a system of interlinked hypertext documents are accessed using particular protocols (i.e., the Hypertext Transfer Protocol and its variants).

Because of the enormous volume of information available via the WWW and the Internet, and because the available information is distributed across an enormous number of independently owned and operated networks and servers, locating desired content on the WWW and the Internet presents challenges.

Search engines have been developed to aid users in locating desired content on the Internet. A search engine is a computer program that receives a search query from a user (e.g., in the form of a set of keywords) indicative of content desired by the user, and returns information and/or hyperlinks to information that the search engine determines to be relevant to the user's search query.

Search engines typically work by retrieving a large number of WWW web pages and/or other content using a computer program called a web crawler that browses the WWW in an automated fashion (e.g., following every hyperlink that it comes across in each web page that it browses). The retrieved web pages and/or content are analyzed and information about the web pages or content is stored in an index. When a user issues a search query to the search engine, the search engine uses the index to identify the web pages and/or content that it determines to best match the user's search query and returns a list of results with the best-matching web pages and/or content. Frequently, this list is in the form of one or more web pages that include a set of hyperlinks to the web pages and/or content determined to best match the user's query.

There are at least two general types of search engines accessible via the Internet: general-purpose search engines and site-specific search engines. As used herein, the term “general-purpose search engine” means a search engine that provides search results that include web pages and/or content (or hyperlinks to web pages and/or content) hosted on at least two different and independent web sites or domains. General purpose search engines attempt to index and provide search results from content distributed across a wide swath of the Internet. Examples of general purpose search engines include Google™, operated by Google, Inc. of Mountain View, Calif.; Yahoo!™, operated by Yahoo!, Inc. of Sunnyvale, Calif.; and Bing™, operated by Microsoft Corp. of Redmond, Wash.

As used herein, the term “site-specific search engine” means a search engine that provides search results that include web pages and/or content (or hyperlinks to web pages and/or content) hosted on only one web site or domain. Site-specific search engines are frequently used by operators of web sites to allow users to find specific web pages or content on their web sites. For example, the web site of an online retailer (or “e-tailer”) may include a site-specific search engine that facilitates a user locating web pages for products sold by the retailer.

SUMMARY

Some embodiments of the invention provide a method, performed by at least one computer, comprising acts of: (A) receiving a query comprising content; and (B) in response to receiving the query, identifying, based at least in part on the content of the query, at least one search engine to which a representation of the query is to be submitted.

Some embodiments of the invention provide a method, performed by a device, comprising acts of: (A) receiving a query from a user; and (B) causing to be displayed results generated based on the query by a plurality of search engines, the plurality of search engines for which results are displayed being determined dynamically in response to the query.

Other embodiments of the invention provide a method, performed by at least one computer, comprising acts of: (A) receiving a query comprising content; (B) in response to the query being received, determining that the content may have at least a first semantic meaning or a second semantic meaning that is different than the first semantic meaning; and (C) identifying a plurality of search engines to which to submit a representation of the query, the plurality of search engines comprising a first search engine identified based on the first semantic meaning and a second search engine identified based on the second semantic meaning.

Still other embodiments of the invention provide a method, performed by at least one computer, comprising acts of: (A) receiving a query; and (B) in response to the query being received, identifying at least one search engine to which to submit a representation of the query, the identifying being based at least in part on consideration received from an operator of the at least one search engine.

Yet other embodiments of the invention provide a method, performed by at least one computer, comprising acts of: (A) receiving a query; (B) in response to the query being received, identifying a plurality of search engines to which to submit a representation of the query; and (C) defining an order of presentation of the plurality of search engines, the defining being based at least in part on consideration received from an operator of at least one of the plurality of search engines.

Some other embodiments of the invention provide a method, performed by at least one computer, comprising acts of: (A) receiving a query from a device, the query comprising content; (B) determining based at least in part on the content of the query that an application is to be launched on the device; and (C) causing the device to launch the application using at least some information determined from the content of the query.

Still other embodiments of the invention provide a method, performed by a device comprising an application, comprising acts of: (A) receiving a free-form query from a user; (B) transferring a representation of the query to at least one computer; and (C) receiving from the at least one computer at least one instruction to launch an application on the device.

Yet other embodiments of the invention provide a method, performed by a device comprising first and second applications, comprising acts of: (A) receiving, from a user, a natural language input to the first application; (B) transferring a representation of the natural language input to at least one computer; (C) receiving from the at least one computer at least one instruction to launch the second application.

Some other embodiments of the invention provide a method, performed by at least one computer, comprising acts of: (A) receiving a query from a device and location data indicating a location of the device, the location data having a level of specificity; (B) in response to the query being received, identifying at least one first search engine to which to submit a representation of the query; (C) determining whether the level of specificity of the location data received in (A) is sufficient for the at least one first search engine; (D) if the level of specificity of the location data is sufficient, instructing the device to issue the representation of the query to the at least one first search engine; and (E) if the level of specificity of the location data is not sufficient, instructing the device to send, to the at least one computer, location data at a greater level of specificity.

BRIEF DESCRIPTION OF DRAWINGS

In the drawings:

FIG. 1 is a Venn diagram showing sets of search results from multiple different search engines;

FIG. 2 is a flow chart of an illustrative process for sending search queries to and receiving search results from multiple search engines, in accordance with some embodiments;

FIG. 3 is a block diagram of a computer environment in which some embodiments may be implemented;

FIG. 4 is a block diagram of a client device that executes an application program for querying multiple search engines and an automated speech recognizer for performing speech recognition on voice search queries, in accordance with some embodiments;

FIG. 5 is a block diagram of a client device that executes an application program for querying multiple search engines and a server that executes an automated speech recognizer and provides speech recognition services for the client device to perform speech recognition on voice search queries, in accordance with some embodiments;

FIG. 6 is a block diagram of a computing environment in which some embodiments may be implemented;

FIG. 7 is a block diagram in which audio data of a voice speech query is recognized using multiple different language models, in accordance with some embodiments;

FIG. 8 is a block diagram of a server that instructs a client device which search engines to query in response to receiving a user-supplied search query, in accordance with some embodiments;

FIG. 9 is a diagram showing Universal Resource Locators (URLs) that may be generated to query multiple search engines, in accordance with some embodiments;

FIG. 10A is a depiction of a display of a client device in which search results from multiple search engines may be displayed;

FIG. 10B is a depiction of a display of a client device in which search results from multiple search engines may be displayed;

FIG. 11 is a block diagram of an illustrative computing device on which aspects described below may be implemented;

FIG. 12 is a flow chart of an illustrative process for identifying the type of information and/or action a user seeks; and

FIG. 13 is a flowchart of an illustrative process for determining whether sufficiently accurate location data is available for use.

DETAILED DESCRIPTION

The inventors have recognized that a given user-specified search query may indicate an interest in searching for any of several different types of information pertaining to the subject of the query. For example, for the search query “Miles Davis,” the user may be interested in obtaining biographical information about Miles Davis, listening to samples of or purchasing Miles Davis' music, and/or social networking with others who are interested in Miles Davis. Historically, to obtain these different types of information, a user may have had to enter the search string “Miles Davis” into three different search engines. For example, the user might have entered this search string into the search engine for an encyclopedia web site to obtain biographical information, into the search engine for a web site that sells music to listen to or purchase music, and into the search engine of a social networking site to connect with others. The inventors have recognized that this process is often time consuming and laborious, as the user must navigate to multiple different web sites (and potentially manage multiple different browser windows or tabs) and enter the same search string repeatedly.

The inventors have also recognized that, because different search engines index web pages and/or content in different ways, index different universes of content, and/or use different algorithms to determine which web pages and/or content best match a particular search query, different search engines may provide different (though possibly overlapping) sets of search results in response to the same search query. This concept is illustrated by the Venn diagram in FIG. 1. Each set depicted in the Venn diagram of FIG. 1 represents the search results from one of four hypothetical search engines (i.e., search engine one, search engine two, search engine three, and search engine four) provided in response to a user-provided search query for the keywords “Miles Davis.” In FIG. 1, set 101 includes results provided from search engine one and includes search results “Q,” “R,” “S,” and “T.” Set 103 includes results provided from search engine two and includes results “R,” “S,” and “Y.” Set 105 includes results provided from search engine three and includes results “S,” “T,” and “X.” Set 107 includes results provided from search engine four and includes result “Z.” As shown in FIG. 1, some search results are included only in one of the sets and, as such, are returned from only one of the four search engines, while other search results are included in multiple of the sets and, as such, are returned from two or more of the search engines.

When a search engine returns a set of search results in response to a search query, the search engine generally returns the results in an ordered list. The list may be ordered by relevance, may be ordered based on money paid for higher positions in the search results, and/or may be ordered based on other criteria. For example, web pages or content that a search engine has determined to be most relevant may be at the top of the list of the results, while web pages or content that the search engine has determined to be less relevant may be farther down in the list of results. As another example, a particular electronics manufacture may pay an electronics e-tailer to list its televisions higher up in the list when users enter search queries including the word “television” into the site-specific search engine for the e-tailer's web site.

The inventors have appreciated that, because different search engines use different algorithms to determine in what order web pages and/or content are to be listed in the search results, even in a situation in which two different search engines include the same web page or piece of content in their search results in response to a particular search query (e.g., “Miles Davis”), any particular web page or piece of content may be at or near the top of the list of results provided by a first of the two search engines (e.g., because the first search engine has determined the web page or piece of content to be particularly relevant in the universe of content in evaluates), but may be further down the list of results provided by a second of the two search engines (e.g., because the second of the two search engines has determined the web page or piece of content to be less relevant in the universe of content it evaluates). Similarly, some results that are listed near the top of the list of search results from the second search engine may be listed much lower in the list of results provided by the first engine. The inventors have recognized that users are more likely to notice and access results that are near the top of the list. Thus, by using only one of the two search engines, a user may not notice or access results that may be highly relevant to the user.

As such, the inventors have recognized that issuing a user-specified search query to only a single search engine increases the chances of some web pages and/or pieces of content that are potentially relevant not being included the search results or being listed low enough in the returned list of search results that the user disregards them. In addition, the inventors have recognized that issuing such a search query to only a single search engine may limit the content that is returned to only the type of content that is search and/or indexed by that search engine, while the user may be interested in other, different types of content pertaining to that search query.

Some conventional web-based software programs, referred to as metasearch engines or search engine aggregators, receive a user-specified search query, issue the search query to multiple search engines, receive results from each of these search engines, remove duplicates, aggregate these search results into a single list, and display this list of aggregated search results to the user. One example of such a metasearch engine is Dogpile™, operated by Infospace, Inc. of Bellevue, Wash. However, the inventors have recognized that these metasearch engines have a number of disadvantages.

First, the search engines that are queried by these metasearch engines include only general-purpose search engines, and do not include any site-specific search engines.

Second, because metasearch engines, in response to a user-specified search query, display to the user a single list of aggregated search results from a number of different search engines, the user has no ability to see which search engine provided which search result, to see how relevant each of the plurality of search engines deemed a particular search result to be, or to look at the full set of results returned from any one particular search engine.

Third, metasearch engines run on one or more servers that receive a user-specified search query issued from a browser or other software application executing on the user's client device and issue the search query from the server(s) to the plurality of search engines that are to be queried. Because a metasearch engine may receive search queries from and provide aggregated search results to a large number of users, the server(s) on which the metasearch engine operates may regularly issue a very large number of search queries to the search engines whose results it aggregates. One consequence of this is that a search engine may receive hundreds of thousands or even millions of search queries daily from the same server or IP address, and may perceive this behavior as a single user issuing an enormous number of search queries to the search engine. Many search engine operators would consider this behavior to be an abuse of the search engine service and/or a potential denial of service attack and would take action to block search queries to their search engines from an IP address issuing such a large number of queries. For this reason, a metasearch engine operator must typically obtain contractual agreements with the search engine operators whose search engine results are aggregated by the metasearch engine to allow the metasearch engine to issue a large number of search queries to these search engines.

Fourth, these metasearch engines do not provide any capability for the user to control to which search engines his or her search query is to be provided. Rather, existing metasearch engines have a fixed set of search engines to which every search query is provided. Thus, the user has no control over which search engines' results are provided in response to a search query and the metasearch engines do not perform any customization of which search engines are queries based on which user issued the search query or based on the content of the user's search query.

While some embodiments of the invention address above-discussed deficiencies of existing metasearch engines, it should be appreciated that not every embodiment addresses all of the deficiencies discussed above, and some embodiments may not address any of these deficiencies. As such, it should be understood that the invention is not limited to embodiments that address all or any of the above-described deficiencies of metasearch engines.

Some embodiments described below are directed to techniques for receiving a user-specified query comprising content and, in response to receiving the query, identifying, based at least in part on the content of the query, at least one search engine to which a representation of the query is to be submitted. For example, some embodiments of the invention are directed to determining, upon receiving a user-specified search query, the type of information which the user seeks, and customizing the search engines to which the query is submitted. This determination may, for example, be based on the content of the user\'s query, the location of the user when the search query was received, the user\'s browsing and/or query history, the browsing and/or query history of other users, any combination of the foregoing, and/or other information. Upon determining the type of information the user seeks, a representation of the query may be submitted to multiple search engines that are selected because they are known to provide access to information of that type. Thus, the search engines to which queries are sent and for which results are displayed may not be static in some embodiments, but rather may be dynamically determined, so that the search engines may vary from query to query for a particular user and/or between different users.

As a simple example, given a user\'s query “Tom Brady,” embodiments of the invention may determine (e.g., based on the content of the query, the user\'s previous search and/or browsing history, etc.) that the user likely seeks information relating to sports. As such, embodiments of the invention may cause a representation of the user\'s query to be submitted to one or more search engines provided by sports-related websites, and cause results generated by the search engine(s) to be presented to the user.

Some embodiments of the invention are not limited to identifying only a single type of information as that which the user seeks, and may determine two or more types. Using the example query “Tom Brady” to illustrate, embodiments of the invention may also acknowledge the possibility that the user seeks celebrity-related information (e.g., based on the query\'s content, the user\'s previous search and/or browsing history, etc.). As a result, some embodiments of the invention may cause a representation of the user\'s query to be submitted to one or more search engines provided by celebrity-related websites, and cause results generated by the search engine(s) to be presented to the user, instead of or in addition to the results generated by sports-related websites.

In seeking to determine the intent of the user, the possibility exists that the type of information sought by the user may be incorrectly identified. To address this risk, some embodiments of the invention may submit a representation of the user\'s query to search engines providing access to other types of information than that which was determined that the user\'s query may be directed to. For example, a representation of the user\'s query may be submitted to one or more general-purpose search engines, and results generated by the search engine(s) may be presented to the user, in addition to the information generated by the more specific search engines (e.g., sports-related and/or celebrity-related websites in the Tom Brady example) which provide information determined to be that in which the user is likely interested. In this way, the system provides the benefit of focused and highly relevant results when it determines the user\'s intent correctly, while having the “safety net” of providing at least some relevant information (e.g., via the general-purpose search engine) if it determines the user\'s intent incorrectly.

While some embodiments that attempt to determine the user\'s intent provide the safety net or fallback option of returning other results returned by additional search engines, not all embodiments are limited in this respect. For example, some embodiments of the invention may only cause results generated by more specific search engines to be presented.

In some embodiments, results generated by the search engines may be arranged based on an assessment of how likely it is that each search engine provides the type of information the user seeks. For example, referring to the Tom Brady example, if it is determined that the user most likely seeks sports-related information, then results generated by one or more search engines provided by sports-related website(s) may be displayed most prominently. If it is determined to also be possible that the user seeks celebrity-related information, then results generated by one or more search engines provided by celebrity-related website(s) may be displayed less prominently than that which is provided by the sports-related websites. If the user\'s query is also submitted to general-purpose search engines (e.g., as a “backup option”), then the results generated by those search engines may be displayed least prominently. It should be appreciated, however, that an arrangement of results generated by various search engines in this manner is optional, and that embodiments of the invention are not limited to displaying results in a manner that is influenced by an assessment of how likely a user seeks a particular type of information.

Displaying some results more prominently than others may be accomplished in any of numerous ways, some non-limiting examples of which are described below. In one example, results from multiple search engines may be displayed concurrently on a display which employs a carousel metaphor, in which each search engine from which results were received occupies a slot. Examples of a carousel are shown in FIGS. 10A-10B, and a commercialized example is the Dragon Search mobile phone application made available by the Assignee of this application, Nuance Communications, Inc. The user may select a particular slot to view results generated by the corresponding search engine. Using the example above to illustrate, the slots corresponding to search engines provided by sports-related websites may be displayed most prominently by, for example, placing the corresponding slots in the center of the carousel, toward the left, or in any other suitable position. The slots corresponding to the search engines provided by celebrity-related websites may, for example, be placed in slots that are deemed less prominent than those corresponding to the sports-related websites, followed by the slots corresponding to general-purpose search engines. In some embodiments, the less prominent slots may be visible when the results are initially presented, whereas in other embodiments they may not, so that the user may need to take an action (e.g., scroll) to view the results for those slots.

As should be appreciated from the foregoing, some embodiments of the invention determine, upon receiving a user\'s query, the type of information the user most likely seeks, and cause information of that type to be presented most prominently to the user, and may cause other types of information (e.g., determined to be less likely than the identified type but still possibly the type the user seeks) to be presented as “backup.” As a result, some embodiments of the invention may be aggressive in presenting specialized information of the type identified to be most likely that which the user seeks, while also allowing for the possibility that the user seeks information that is not limited to any particular type.

FIG. 2 depicts an example process 200 that may be performed to determine, upon receiving a user\'s query, the type of information that the user likely seeks, and to present that type of information (e.g., alone or among one or more other types of information) to the user. Process 200 may, for example, be performed by an application program executing on a client device 250 employed by the user and/or a computer 275 that is external to the client device (e.g., in communication with client device 250 via a communication medium, not shown). Client device 250 may be any suitable type of computing device having hardware capable of executing an application program. Examples of client devices that may be used include a laptop or desktop personal computer, a personal digital assistant (PDA), a mobile phone (e.g., a smartphone), a server computer, and/or various other types of computing devices. External computer 275 may also include any suitable type of computing device, such as a laptop or desktop personal computer, server computer, and/or other type of computing devices. Collectively and/or individually, the client device and external computer may provide speech recognition capability, so that a user-specified query may be submitted via voice input. However, embodiments of the invention are not limited to systems having speech recognition capability, and are not limited to systems comprising mobile telephones or smartphones.

Process 200 begins in act 201, wherein an application program on client device 250 receives a user-specified query (e.g., via voice and/or text input). As discussed in greater detail below, the query may be received in any of numerous ways via any of numerous user interfaces. In the example process shown in FIG. 2, the application program executing on the client device causes a representation of the user-specified query to be transmitted to external computer 275 in act 203, which receives it in act 205 and identifies the type of information and/or action which the user seeks in act 207. As described in detail below, the representation of the query can take any of numerous forms, as can the identification of the information and/or action specified by the query. External computer 275 generates information (e.g., comprising search queries to be submitted to the identified search engines) in act 209, and transmits the information to the application program executing on the client device in act 211. After receiving the transmitted information in act 213, the application program executing on client device 250 processes the information (e.g., by issuing search queries to each identified search engine) in act 215. If act 215 includes issuing search queries to search engines, then the application program executing on client device 250 receives results generated from those queries in act 217. Results are displayed to the user in act 219. Example process 200 then completes.

It should be appreciated that example process 200 represents only one manner of implementing embodiments of the invention. For example, not all embodiments of the invention are limited to employing a client device executing an application program and an external computer, as numerous other system configurations may alternatively be employed. For example, some embodiments of the invention may receive a user-specified query, identify search engines making available the types of information the user may seek, generate search queries and display results therefrom, all on a single device. In another example, some embodiments of the invention may employ multiple computers external to the client device to identify search engines making available the types of information the user may seek and generate search queries. Variations on example process 200 may be performed by any of numerous component configurations, as embodiments of the invention are not limited in this respect.

Further, it should be appreciated that numerous variations on example process 200 are possible. For example, some variations may include one or more of acts 201-219 being performed in a different sequence than that which is described herein, may omit one or more of acts 201-219, and/or may include acts not described. Embodiments of the invention are not limited to being implemented in any particular manner.

The acts comprising example process 200 are described in detail below.

I. Receiving a User-Specified Search Query

In act 201, an application program executing on client device 250 receives user input specifying a search query. A search query may take any suitable form, and may not be bound by a predefined syntax. For example, a search query may be free-form, and comprise any information susceptible to representation via a string of one or more characters. For example, a search query may comprise a name of a person (e.g., “Tom Brady”), place (e.g., “Saratoga Springs”), thing (e.g., “New York Stock Exchange”), any combination thereof (e.g., “ski shops in Park City”), and/or otherwise convey a user\'s desire to search for information on one or more particular topics or concepts.

User input defining a search query may be received in any of a variety of ways and in any of a variety of formats. For example, in some embodiments, the user input may be received as text and may be received via an input device (e.g., a keyboard, touch screen, mouse, and/or any other input device) integrated into or coupled to the client device. In some embodiments, user input may be received as audio via a microphone that is integrated into or coupled to the client device. For example, a user may speak a search query into the microphone and the application program may receive the user\'s voice input as audio data. In some embodiments, the client device may provide a number of different options from which the user may select to provide a search query. For example, in some embodiments, the client device may permit the user to input the query either in voice format (e.g., by speaking the query into a microphone) or in text format (e.g., by typing the query on a keyboard or a touchscreen).

In the example process 200, a representation of the user-specified query received in act 201 is transmitted to external computer 275 in act 203, and received by the external computer in act 205. In embodiments wherein the query received in act 201 is specified via voice input, automated speech recognition may be performed on audio data representing the voice input to obtain a recognition result (e.g., in textual form) that may be used to identify one or more search engines making available the type(s) of information the user seeks, and to generate queries to be submitted to those search engines. However, not all embodiments of the invention are limited to performing automated speech recognition. For example, one or more of the search engines to be queried may have a speech interface, so that audio data may be provided to such search engines instead of providing a text search query based on automated speech recognition of the audio. Embodiments of the invention are not limited to any particular implementation.

If performed, automated speech recognition may be conducted by client device 250, external computer 275, or a combination thereof. Automated speech recognition techniques are known, and any suitable automated speech recognition technique may be used to obtain a recognition result for the audio data.

As shown in FIG. 4, in some embodiments, the client device may execute an automated speech recognizer, which is a software program that performs automated speech recognition on audio data, and the application program may use the automated speech recognizer to obtain a recognition result of the audio data. In FIG. 4, client device 401 executes the application program 403 that performs at least some of the above-described acts of process 200 (e.g., those performed by the client device) and an automated speech recognizer 405. When application program 403 receives user-supplied audio data 407, it may pass the audio data to automated speech recognizer 405. Automated speech recognizer 405 may perform automated speech recognition on the audio data to obtain a recognition result 409 and may return recognition result 409 to application program 403 for use in formatting queries for the speech engines.

The inventors have recognized that some client devices may not have sufficient computing resources to execute an automated speech recognizer that operates with a desired level of accuracy and/or efficiency. That is, for example, a client device may not have sufficient memory to store acoustic models, language models, grammars, and/or other components of such a speech recognizer and/or may not have a processor powerful enough to perform automated speech recognition at a desired level of efficiency. This may occur in any number of situations, such as when the client device is a handheld device, such as a PDA or mobile phone.

Thus, in some embodiments, automated speech recognition on audio data that originates from a user may be performed by the external computer. As shown in FIG. 5, client device 501 executes application program 503 that receives voice input supplied by a user in the form of audio data. Automated speech recognition may be performed on a representation of the audio data by an automated speech recognizer 507 executing on a server 505. That is, automated speech recognition may be performed on the full audio data, or some processed representation of it. Thus, as shown in FIG. 5, in some embodiments, application program may send the user-supplied audio data 509 to automated speech recognizer 507 executing on server 505. The audio data 509 may be sent to the automated speech recognizer in any of a variety of possible ways, and may be processed on the client device and/or on the server before being passed to the speech recognizer. For example, in some embodiments, the user-supplied audio may be captured by a microphone and analog-to-digital conversion may be performed (on the client or server) on the captured microphone audio to generate digital audio data. This digital audio data may be supplied to automated speech recognizer 507. In some embodiments, some processing of the digital audio data may be performed on the client and/or server prior to sending this data to the automated speech recognizer. For example, the digital data may be compressed or some pre-processing may be performed to extract parameters of the audio that are useful in automated speech recognition. The extracted parameters may be passed to automated speech recognizer 507 as the representation of the audio data or may be sent along with other audio data. In embodiments that use compression, any of a variety of possible compression algorithms may be used. For example, in some embodiments, the Speex™ codec may be used.

Thus, it should be appreciated that, as used herein, the term “audio data” may refer to an analog audio signal, a digital representation of an analog audio signal, a compressed representation of such a digital or analog signal, a set of features or parameters extracted from the digital or analog signal, any combination of the foregoing, and/or other information.

Automated speech recognizer 507 may perform automated speech recognition on audio data 509 and return to application program 503 a response 511 that comprises results from audio data 509. The results may take any suitable form. As one non-limiting example, the results may comprise a text recognition result obtained by automated speech recognizer 507 from performing automated speech recognition on the audio data.

In the example of FIG. 5, client device 501 is depicted as communicating directly with server 505. It should be appreciated that this depiction is provided merely to facilitate an understanding of the types of information sent between client device 501 and server 505, and that client device 501 may communicate with server 505 in any of a variety of ways and via any suitable communication medium, including, for example, via the Internet.

In the example of FIG. 4, automatic speech recognition on audio data received at a client device is performed on the client device, and in the example of FIG. 5, audio data is provided from the client device to a server and the server performs automatic speech recognition. However, in some embodiments, automatic speech recognition may be performed partially by the client device and partially by the server. For example, a portion of an ASR process may be performed on the client and a portion may be performed on the server. This may be done, for example, to enable the client device to perform a portion of the speech recognition process, but leave computationally intensive portions for the server.

In some embodiments, one or more servers (like server 505) may provide automated speech recognition services for multiple client devices. FIG. 6 shows a computing environment in which each of a plurality of users 601a, 601b, . . . 601n operates a client device 603a, 603b, . . . , 603n. Each client device 603 may execute an application program that sends search queries via communication medium 607 to search engines 609a, 609b, 609c, . . . , 609n, and displays the results of these search queries to its respective users (e.g., by executing some of the acts of process 200 of FIG. 2). Communication medium 607 may comprise any suitable communications infrastructure, employing any suitable communication protocol(s), including wired and/or wireless communications protocols. For example, communication medium 607 may comprise one or more intranets, local area networks (LANs), wide area network (WANs), the Internet, or any combination thereof.

Server 605 may provide automated speech recognition services to support these searches when the user enters a query audibly. Thus, in embodiments in which client devices 603 receive user-supplied search queries in the form of audio data, the client devices 603 may send some or all of the audio data via communication medium 607 to server 605 to have automated speech recognition performed on the audio data, and may receive a recognition result and/or other information (e.g., search queries and an identification of search engines to issue them to) via communication medium 607 based on the recognition result in response.

In some alternate embodiments, server 605 may provide other functionality (either instead of or in addition to ASR services) to facilitate issuance of search queries from a client device 603 to one or more of search engines 609. Some examples of this additional functionality are described in detail below. For the sake of simplicity, in the example of FIG. 6, only one server 605 is depicted. However, it should be understood that, as is conventional for Internet servers that receive a high volume of traffic, any number of servers may be used to provide the functionality of server 605 that is described herein and the traffic load from client devices 603 may be balanced across these servers using any of a variety of load balancing techniques. In addition, the functionality of the servers may be distributed in any suitable way, so that each server need not perform the same functions and may be arranged to share the functions performed by the set of servers collectively.

In embodiments in which a user supplies a search query by voice (i.e., in the form of audio data), a number of techniques may be used to facilitate automated speech recognition of the audio data to generate search queries. These techniques may be employed in embodiments in which automated speech recognition is performed entirely on the client device, in embodiments in which automated speech recognition is performed entirely on a computer external to the client device, and in embodiments wherein ASR is distributed between the client device and an external computer.

In some embodiments, one or more language models used by the automated speech recognizer in performing speech recognition on the audio data of the user-supplied search query may be selected based on the search engine(s) which are to be queried. For example, as shown in FIG. 7, in some embodiments, the automated speech recognizer may perform recognition on the same audio data using multiple different language models to yield multiple different recognition results. In FIG. 7, to speech recognition is performed on audio data 701 using language models 703a, 703b, . . . , 703n to generate recognition results 705a, 705b, . . . , 705n. Depending on the content of audio data 701 and the differences between language models 703, each of recognition results 705 may be the same, each may be different from the others, or some may be the same and some may be different. One or more of language models 703 may be associated with a particular search engine and the recognition result generated using a language model that is associated with a particular search engine may serve as the basis for the search query that is issued to that search engine.

Thus, for example, in some embodiments, a language model that has been trained on the content that is indexed by a particular search engine may be used to generate the recognition result that serves as the basis for the query that is issued to that search engine. This results in a query that is more likely to yield relevant search results for that search engine.

This technique may be particularly useful when issuing queries to site-specific search engines that index a limited amount of content that is generally directed to a particular subject area. For example, an online shoe retailer may have a site-specific search engine for its e-commerce web site. As such, the content of this web site may include shoe-specific terminology and phrases, such as the name of shoe brands and models, descriptions of shoe properties, and various other shoe-specific information. By training the language model used by a speech recognizer on the content of this web site, the recognizer is more likely to accurately recognize speech that includes the shoe-specific terminology and phrases.

Thus, for example, if a user-supplied search query in the form of audio input is received and is to be issued to both a site-specific search engine and a general-purpose search engine, the audio input may be recognized using a first language model (e.g., a language model that has been trained on content related to the content indexed by the site-specific search engine) to generate a recognition result that serves as the basis for the query to be issued to the site-specific search engine and may also be recognized using a second language model (e.g., that has been trained on more general content), different from the first language model, to generate a recognition result that serves as the basis for the query to be issued to the general-purpose search engine.

In the example described above, the recognition result that is obtained by performing speech recognition using a language model that is associated with a particular search engine serves as the basis for the search query that is issued to that search engine. However, in some embodiments, rather than issuing recognition results obtained using different language models to different search engines, multiple recognition results may be obtained using different language models and the recognition result with the highest score or confidence value may serve as the basis of a search query that is issued to multiple search engines. For example, in FIG. 7, each of recognition results 705 may be generated by performing automatic speech recognition on audio data 701 using a corresponding language model. In some embodiments, one of recognition results 705 may be selected to be the result that serves as the basis for generating a search query to be issued to multiple search engines. This selection may be made in any suitable way. For example, as discussed above, the recognition result that has the highest score or confidence value as a result of the speech recognition may be selected, or any other suitable criterion or combination of criteria may be used.

In situations in which speech recognition is performed on the same audio data using multiple language models, performing multiple separate and independent speech recognition processes with each of the different language models may increase the latency in obtaining recognition results and/or may increase the amount of processing resources used in performing speech recognition.

Thus, in some embodiments, rather than performing an entire independent speech recognition process on the audio data for each language model, one generalized language model may be used to create a lattice of hypothesized word sequences, and this lattice may be re-scored using one or more selected language models to yield recognition result(s) particularly for the selected language model(s). In this way, the time and processing resources consumed in generating the recognition results may be reduced because the same lattice is used in producing each recognition result and thus only needs to be generated once.

In the example of FIG. 7, audio data 701 is shown as being provided directly to each of language models 703. This depiction is provided merely to facilitate an understanding of how multiple language models may be used, and it should be understood that, in practice, the language models may not operate directly on audio data. For example, a lattice with multiple hypothesized sequences of words may be obtained from audio data using an acoustic model and a general language model, and specialized language models 703 may be used to rescore the general lattice and obtain a sequence of words appropriate to a specific search engine.

In some embodiments, rather than using multiple different language models in generating search queries to be provided to search engines, one language model may be selected from a plurality of available language models. For example, in some embodiments, one language model may be identified as being well-suited for recognizing a particular voice query, and that language model may be used in recognizing the query.

Any of a variety of possible criteria or combinations of criteria may be used to select a language model to be used in recognizing a particular voice query. For example, in some embodiments, the content of the query may be used as a basis for selecting a language model to be used. For example, a two-pass process may be used whereby in a first pass automatic speech recognition of a voice query or a portion of a voice query is performed using a general language model that doesn\'t have a specialized vocabulary. Such a language model may recognize some of the words in the query, but may not accurately recognize all of the words. The recognition result obtained from the first-pass may be used to classify the query based on topic or category. Any suitable classification technique may be used, and a variety are known.

Based on the classification, a language model that has a specialized vocabulary directed to the identified topic or category may be selected. After selection of the language model, a second-pass of automatic speech recognition may be performed using the selected language model. For example, if after the first-pass it is determined that the voice query includes word such as “directions,” “street,” or “road,” it may be determined that voice query is map-related query, and a map-related language model (e.g., a language model that includes city, street, and place names for the United States and/or other countries) may be selected and used in the second-pass.



Download full PDF for full patent description/claims.

Advertise on FreshPatents.com - Rates & Info


You can also Monitor Keywords and Search for tracking patents relating to this Methods and apparatus for delivering information of various types to a user patent application.
###
monitor keywords

Browse recent Nuance Communications, Inc. patents

Keyword Monitor How KEYWORD MONITOR works... a FREE service from FreshPatents
1. Sign up (takes 30 seconds). 2. Fill in the keywords to be monitored.
3. Each week you receive an email with patent applications related to your keywords.  
Start now! - Receive info on patent apps like Methods and apparatus for delivering information of various types to a user or other areas of interest.
###


Previous Patent Application:
Menu configuration
Next Patent Application:
Providing customization of context based menus
Industry Class:
Data processing: presentation processing of document
Thank you for viewing the Methods and apparatus for delivering information of various types to a user patent info.
- - - Apple patents, Boeing patents, Google patents, IBM patents, Jabil patents, Coca Cola patents, Motorola patents

Results in 0.91765 seconds


Other interesting Freshpatents.com categories:
Medical: Surgery Surgery(2) Surgery(3) Drug Drug(2) Prosthesis Dentistry  

###

Data source: patent applications published in the public domain by the United States Patent and Trademark Office (USPTO). Information published here is for research/educational purposes only. FreshPatents is not affiliated with the USPTO, assignee companies, inventors, law firms or other assignees. Patent applications, documents and images may contain trademarks of the respective companies/authors. FreshPatents is not responsible for the accuracy, validity or otherwise contents of these public document patent application filings. When possible a complete PDF is provided, however, in some cases the presented document/images is an abstract or sampling of the full patent application for display purposes. FreshPatents.com Terms/Support
-g2-0.2652
Key IP Translations - Patent Translations

     SHARE
  
           

stats Patent Info
Application #
US 20130019202 A1
Publish Date
01/17/2013
Document #
13221332
File Date
08/30/2011
USPTO Class
715810
Other USPTO Classes
707706, 707E17108
International Class
/
Drawings
14


Your Message Here(14K)


Graphics
Search Engine
Graph


Follow us on Twitter
twitter icon@FreshPatents

Nuance Communications, Inc.

Browse recent Nuance Communications, Inc. patents

Data Processing: Presentation Processing Of Document, Operator Interface Processing, And Screen Saver Display Processing   Operator Interface (e.g., Graphical User Interface)   On-screen Workspace Or Object   Menu Or Selectable Iconic Array (e.g., Palette)