Generation of automated reports and matching using online surveys and collaborative filtering

This application is a continuation of U.S. patent application Ser. No. 10/748,968, filed Dec. 29, 2003, now allowed, which claims the benefit of Provisional Application Ser. No. 60/436,809, filed Dec. 27, 2002, each of which is incorporated herein in its entirety, by this reference thereto.

1. Field of the Invention

The invention relates to correlating statistical records. More particularly, the invention relates to correlating compensation records to unique individual profiles.

2. Background of the Invention

Today, many reports are available that allow a user to find, read, purchase, or otherwise acquire reports on worker compensation. Most often these reports indicate average pay rates by industry, job type, locale, and sometimes they report more specific information about a particular industry or job, such as bonuses, stock options, average workweek, or immigration status, among other things. To create such compensation reports two approaches are typically used. One such approach is for a human analyst to research and find a statistically valid number of individuals with like characteristics, and devise a suite of compensation reports. This process is tedious, labor intensive, and often expensive. For a truly detailed report the analyst must be relied on to do substantial investigation and synthesize and apply this information to the case at hand. Compensation consultants with years of experience and resources can generally accurately profile an individual\'s worth in the market place, however such an analysis is extremely specialized and out of the reach of the typical consumer. Simpler and less costly reports are available but they are generally broadly classed and offer little utility.

The majority of software-based analysis provides a less expensive alternative but yields correspondingly limited information. Compensation services using current computer analysis programs generally gather data using some form of questionnaire and then feed the appropriate data into a computer database or spreadsheet. More typically, generalized data, such as from the US Bureau of Labor and Statistics, is used as a base and then extrapolated based on region and date, and often combined with third party surveys. Typically, a computer then is instructed to run an analysis of the data to provide statistical information, such as averages, medians, and standard deviations on pre-determined groups of people. However the information provided is not unique to an individual, but instead, is a conglomeration of data that the program determines best represents the individual. Because the categorization of the individual is based solely on a limited, predetermined set of responses to the questionnaire, it offers little to no opportunity for evaluating unique characteristics. For example, an automated compensation service may categorize and calculate data showing that the average yearly salary of a “Computer Programmer Level 3” in Washington state is $64,250. This may or may not be applicable to a “Senior Application Software Engineer” with “10 years of experience” and special training in the skill “C++,” but because the closest answer describing the Senior Application Software Engineer\'s position in the initial survey was a “Computer Programmer Level 3,” the Senior Application Software Engineer has thus been categorized ineffectively, which removes any unique abilities that he may possess.

The Senior Application Software Engineer reading the aforementioned report cannot be sure how closely the published report figures apply to himself individually. There are a multitude of factors that affect any one individual\'s job compensation. The current generalized reporting methods for compensation reports, cannot and do not incorporate factors that provide for an accurate job comparison and compensation analysis for individual users. Today\'s methods require the user to gauge or self-approximate themselves to a group of people being reported. Typically, such approximations are grouped by a specific job title that a human compensation analyst predetermines when creating a report, or when designing a computer service that eventually generates the report. This grouping is generally not an exact match with the user\'s actual job title and responsibilities and often has little applicability to the users individual qualities. For example, the compensation analyst might have created a report for an isolated group called Computer Programmer Level 3. For individuals who possess the same characteristics as the data files used to create this group, the reports generated from such a compensation analysis are reasonably accurate. However, for individuals possessing unique capabilities, experiences, skills, or talents, the reports are essentially useless. The data are by definition misapplied because any differences in the compared data are arbitrarily reflected in the compensation report. This introduces doubt on the user\'s part as to how closely he can trust the report\'s applicability.

Current compensation analysis techniques do not provide users with affordable, accurate, and personalized compensation reports. Job specific variables, critical to the accurate assessment of an individual\'s worth, are not correctly identified or uniformly applied. Furthermore individuals within a particular field are unaware of the value of certain, often easily obtainable, qualifications. There is a need, therefore, for an apparatus and method that provides online compensation reports using a more flexible survey system to produce dynamic profiles based on unique individual attributes, and to provide automated comparisons and reports that account for these attributes.

In one embodiment of the invention, profiles are used to produce individualized compensation reports. A survey engine is used to produce profiles of individuals that identify the individuals\' unique characteristics. The survey engine incorporates a collaborative filtering engine that determines appropriate questions to ask the user during the survey, and also provides suggested possible answers. Additionally, the system allows for the use of open-text questions that allow for new answers to be input by the user, without the prior need for an administrator to pre-define the possible values for the system, as is typical in prior art. The system incorporates affinity groups around profile attributes (question answers), providing a basis for gauging similarity of profiles for various comparison and aggregation purposes. A collaborative filtering engine is incorporated, which is both periodically modified by an administrator, and also tuned by users themselves based on their actions and responses. New affinity groups (associations of profiles) are incorporated by the survey engine to suggest new questions and possible answers in a survey. Additionally, some affinity groups are generated automatically by the system, and finally by users themselves to create new, interesting relationships among profiles.

The collaborative filtering engine enables the capture of profile attributes that are targeted compensation variables concerning a profile. The system automatically incorporates new profiles into existing affinity groups, which allows more targeted survey questions and possible answers to be determined, without requiring constant human training or intervention.

Without a collaborative filtering system, the system would be unable to administer surveys accurately for users who do not fit into pre-defined categories because it is currently impossible to categorize every occupational variation. Because the system allows for open-text answers to questions, and because the system provides for different questions to be automatically determined and asked for differing types of user job profile, the collaborative filtering system, along with affinity groups and other requirements, is employed. Because the collaborative filtering system allows for the system to make educated guesses within defined constraints, the system can handle new categorizations more effectively than a system wholly defined by a human administrator. Additionally, in this system, the administrator defines constraints that prohibit the survey from asking obviously wrong or out of place questions. An example of a constraint is requiring that if the user does not answer any of the suggested questions, a default question is always asked.

One aspect of using a collaborative filtering system to define a survey is that the system accommodates a much larger population of data using a more targeted survey. Previous implementations of compensation surveys relied on a smaller sample size, a broad survey with generalized questions, or a larger base of analysts to design and conduct surveys and categorize the data. Thus, the survey was constrained purely by the human resources needed to conduct it. The invention described herein is not constrained as such and requires far fewer human resources to conduct such a detailed survey across many different job categories.

Another aspect of the invention comprises the ability to search through the data that has been collected by the survey engine, using detailed search criteria defined by a search definition document. The document applies a scoring and filtering mechanism that returns the most appropriate set of profiles based on the user\'s goal for the analysis. The search document is useful because it allows an administrator to define natural relationships between profile attributes, such as those that apply routinely to the realm of compensation analysis of various classes of profiles. The document is then interpreted by a software algorithm, and used during the retrieval of relevant profiles, which can then be used for tallying and reporting. Multiple search definition documents can be created quickly, each one for a different goal. For example, one search goal may be to weight skills and certifications very highly. The results are useful for analyzing how people with similar technical skills are compared to each other. Another search definition document could weight experience and education higher, which is useful in seeing how a user compares to those profiles who have similar experience and education levels. A third example could be to hold location constant, and compare a user to all other profiles with matching attributes in the same location.

Another aspect of the invention comprises the automated ability to summarize and present the results of a profile search in a format that is useful to a human for compensation comparisons. A Chart is defined as a series of values, such as skills paired, with a series of measures, such as average salary, median salary, standard deviation, etc. A sample chart could be called “Average Salary By Skill” and it would list each Skill along with an associated average salary. A Report is defined as a series of charts combined to provide an overall picture and analysis for a user. For instance, consider a report that aims to discover how a user compares with regard to skills and experience in similar jobs. This report incorporates many charts which are combined into a format that gives a user a good analysis and understanding of the results of the user\'s search goal.

The prior art approaches are unable to automate the selection of charts within a compensation report, such as determining if “Average Salary by Practice Area” is applicable to a report presented for a particular user profile. It may only be applicable to a Lawyer, for instance. The prior art typically requires that the administrator know in advance, every chart that would be relevant for a particular user. This restricts known approaches in one of two ways: either they limit the number of charts available, such that all charts are available to all compensation reports, regardless of their relevance, or they predefine different compensation reports by industry or job category, which would require a large amount of labor. In contrast, the invention described herein determines if a chart is relevant to a user based on the user\'s own profile, and displays it, and also only shows it if enough data exists for it to be valid statistically. An advantage of the invention herein is provided because there may be thousands of different charts of many different types, based on many different attributes and measures, and only a subset of them may apply to a particular users\' compensation analysis. For instance, a Lawyer may wish to see a report concerning “Average Bonus by Number of Hours Billed per Year,” but this report may not be applicable to Teachers or CEOs. In the prior art, each of these compensation reports must be compiled by a human analyst, created at expense, or details are ignored, leaving only the most common charts in the compensation report, such as “Average Salary by Occupation and Location,” which are often the least useful to an individual trying to compare his compensation against his peers. The invention herein allows for a more targeted and relevant compensation report by focusing in an automated, scalable fashion, on attributes that are unique to a particular user, in addition to the attributes that are most common to all users.

The invention may also be used to match individual profiles to resumes because the profiles described herein are typically a subset of data gathered for a resume. For example, a user entering their profile into the system described herein could be easily matched to resumes using the same mechanisms employed to match against other profiles.