| Learning control apparatus, learning control method, and computer program -> Monitor Keywords |
|
|
  Learning control apparatus, learning control method, and computer programUSPTO Application #: 20060190156Title: Learning control apparatus, learning control method, and computer program Abstract: A learning control apparatus for an autonomous agent including a functional module having a function of multiple inputs and multiple outputs, the function receiving at least one variable and outputting at least one value, includes an estimating unit for estimating a causal relationship of at least one variable, a grouping unit for grouping at least one variable into a variable group in accordance with the estimated causal relationship, a determining for determining a behavior variable corresponding to each of the variable groups, and a layering unit for layering, in accordance with the variable group and the behavior variable, the function corresponding to each variable group, the function receiving the variable grouped into the variable group and outputting the behavior variable. (end of abstract) Agent: Oblon, Spivak, Mcclelland, Maier & Neustadt, P.C. - Alexandria, VA, US Inventors: Kenichi Hidai, Kohtaro Sabe USPTO Applicaton #: 20060190156 - Class: 701058000 (USPTO) Related Patent Categories: Data Processing: Vehicles, Navigation, And Relative Location, Vehicle Control, Guidance, Operation, Or Indication, Transmission Control, Adaptive Control The Patent Description & Claims data below is from USPTO Patent Application 20060190156. Brief Patent Description - Full Patent Description - Patent Application Claims
CROSS REFERENCES TO RELATED APPLICATIONS [0001] The present invention contains subject matter related to Japanese Patent Application JP 2005-047491 filed in the Japanese Patent Office on Feb. 23, 2005, and Japanese Patent Application JP 2005-169458 filed in the Japanese Patent Office on Jun. 9, 2005, the entire contents of which are incorporated herein by reference. BACKGROUND OF THE INVENTION [0002] 1. Field of the Invention [0003] The present invention relates to a learning control apparatus, a learning control method, and a computer program. More particularly, the present invention relates to a learning control apparatus, a learning control method, and a computer program for an autonomous learning agent having multiple dimensional variables (sensor input, internal state, and motor output), the autonomous learning agent estimating a causal relationship between variables based on learning of a predictor, and automatically determining the number of controllers, functions, and input and output variables based on the estimated causal relationship in order to automate modularization and layering of the controllers. [0004] 2. Description of the Related Art [0005] Structure of an autonomous agent using reinforcement learning is disclosed by Richard S. Sutton, and Andrew G. Barto in the book entitled "Reinforcement Learning: An Introduction" MIT Press, 1998. Experience-reinforced autonomous agent is disclosed by Jun Tani in the paper entitled "Learning to generate articulated behavior through the bottom-up and the top-down interaction processes", Neural Networks, Vol. 16, No. 1, pp. 11-23, 2003. In these disclosed techniques, input and output variables of a learner are manually selected by humans who take into consideration a task to be solved and an expected behavior. [0006] As for multi-degree-of-freedom agent, if a task and input and output variables are determined during design phase, learning capability of the agent is limited from the design phase. The known techniques are thus subject to serious problem in the construction of an open-ended autonomous agent. [0007] If all conceivable sensor and motor variables are used as inputs and outputs to solve the manual selection problem, performance in individual task and expected behavior is affected. This is well known as curse of dimensionality in the field of machine learning (as disclosed by R. E. Bellman, in the book entitled "Dynamic Programming" Princeton University Press, Princeton. 6: 679-684). [0008] To overcome this problem, an autonomous agent is segmented in a plurality of functional modules in learning. However, this process leads to two new problems which do not exist if learning is performed with a single functional module. [0009] A first problem is how to determine the number of functional modules and the degree of freedom of each module (a quantity for determining how complex structure one module is allowed to have). A second problem is how a link between modules is determined. [0010] MOSAIC disclosed in Japanese Unexamined Patent Application Publication No. 2000-35804 is interesting in terms of function modularization. However, each module needs to handle all variables as inputs and outputs, and MOSAIC thus fails to overcome the two new problems. To overcome the two new problems, humans need to design beforehand a link with the functions of the modules. MOSAIC remains unchanged from a design approach incorporated in the known autonomous agent. SUMMARY OF THE INVENTION [0011] To overcome the drawback of MOSAIC, it is contemplated to automatically learn the function of each functional module and the link thereof. In such a case, a method of determining the number of modules and the complexity of each module is not known, and there is room for human intervention. Any case that a plurality of tasks and a plurality of expected behaviors are additionally learned in a non-destructive fashion has never been reported. [0012] According to brain science, neural science, and psychology, humans are believed to have modularity of function (localized function). There is an argument that some modularity and layer structure be introduced in learning of artificial autonomous agent. No complete methods to link and integrate modules and layers have not been established as disclosed by G. Taga in the book entitled "Dynamic design of brain and body, and non-linear dynamics and development of motion and perception," Kanekoshobo, 2002. [0013] It is thus desirable to provide an autonomous learning agent having multiple dimensional variables (sensor input, internal state, and motor output), the autonomous learning agent estimating a causal relationship between variables based on learning of a predictor, and automatically determining the number of controllers, functions, and input and output variables based on the estimated causal relationship in order to automate modularization and layering of the controllers. [0014] A learning control apparatus of one embodiment of the present invention includes an estimating unit for estimating a causal relationship of at least one variable, a grouping unit for grouping at least one variable into a variable group in accordance with the estimated causal relationship, a first determining unit for determining a behavior variable corresponding to each of the variable groups, and a layering unit for layering, in accordance with the variable group and the behavior variable, the function corresponding to each variable group, the function receiving the variable grouped into the variable group and outputting the behavior variable. [0015] The learning control apparatus further includes a second determining unit for determining a state variable corresponding to each variable group. [0016] A learning control method of another embodiment of the present invention includes steps of estimating a causal relationship of at least one variable, grouping at least one variable into a variable group in accordance with the estimated causal relationship, determining a behavior variable corresponding to each of the variable groups, and layering, in accordance with the variable group and the behavior variable, the function corresponding to each variable group, the function receiving the variable grouped into the variable group and outputting the behavior variable. [0017] A computer program of another embodiment of the present invention includes steps of estimating a causal relationship of at least one variable, grouping at least one variable into a variable group in accordance with the estimated causal relationship, determining a behavior variable corresponding to each of the variable groups, and layering, in accordance with the variable group and the behavior variable, the function corresponding to each variable group, the function receiving the variable grouped into the variable group and outputting the behavior variable. [0018] In accordance with embodiments of the present invention, the causal relationship of at least one variable is estimated. At least one variable is grouped into variable groups in accordance with the estimated causal relationship. The behavior variable corresponding to the variable group is determined. The function corresponding to each variable group is layered in accordance with the variable group and the behavior variable, the function receiving the variable grouped into the variable group and outputting the behavior variable. [0019] In accordance with embodiments of the present invention, an autonomous learning agent having multiple dimensional variables (sensor input, internal state, and motor output), estimates a causal relationship between variables based on learning of a predictor, automatically determines the number of controllers, functions, and input and output variables based on the estimated causal relationship, and automates designing of modularization and layering of the controllers. BRIEF DESCRIPTION OF THE DRAWINGS [0020] FIG. 1 illustrates an autonomous agent; Continue reading... Full patent description for Learning control apparatus, learning control method, and computer program Brief Patent Description - Full Patent Description - Patent Application Claims Click on the above for other options relating to this Learning control apparatus, learning control method, and computer program patent application. ###  How KEYWORD MONITOR works... a FREE service from FreshPatents1. Sign up (takes 30 seconds). 2. Fill in the keywords to be monitored. 3. Each week you receive an email with patent applications related to your keywords. Start now! - Receive info on patent apps like Learning control apparatus, learning control method, and computer program or other areas of interest. ### Previous Patent Application: Device for controlling an engine or a gearbox Next Patent Application: Brake interface module Industry Class: Data processing: vehicles, navigation, and relative location ### FreshPatents.com Support Thank you for viewing the Learning control apparatus, learning control method, and computer program patent info. IP-related news and info Results in 2.9456 seconds Other interesting Feshpatents.com categories: Accenture , Agouron Pharmaceuticals , Amgen , AT&T , Bausch & Lomb , Callaway Golf |
||
