| Development environment for data transformation applications -> Monitor Keywords |
|
|
  Development environment for data transformation applicationsUSPTO Application #: 20080052687Title: Development environment for data transformation applications Abstract: An environment and method for developing dataflow applications where a dataflow transformation is encapsulated into a dataflow graph expressed as a reusable map component using a host development language such as Java®, and the dataflow graphs are linked together via ports. A scalar map component performs a specific data transformation and a composite map component is built from a number of other map components typically representing a family of data transformations. Map components have interface and implementation properties which assist in developing customized, hierarchical data transformation logic. The ports are polymorphic and can be specific, generic, or composite data types, to allow linking of map components which compile to generate a dataflow pattern. A map component may employ dynamic logic where internal sub-graphs or links are added or properties changed based on information received from the compiler, such as port types, design properties, or static data transformation. (end of abstract)
Agent: Daffer Mcdaniel LLP - Austin, TX, US Inventors: Agustin Gonzales-Tuchmann, Larry Lee Schumacher, Laurence Tobin Yogman, Paul C. Dingman USPTO Applicaton #: 20080052687 - Class: 717140 (USPTO) The Patent Description & Claims data below is from USPTO Patent Application 20080052687. Brief Patent Description - Full Patent Description - Patent Application Claims
BACKGROUND OF THE INVENTION [0001]1. Field of the Invention [0002]The present invention relates to a system and method for managing data which includes data transformation, such as data warehousing, data analysis or similar applications. In particular, the invention relates to the synthesis from dataflow graphs of computationally efficient executable applications well suited for parallel execution. [0003]2. Description of the Related Art [0004]The following descriptions and examples are not admitted to be prior art by virtue of their inclusion within this section. [0005]Business processes are collecting ever-increasing amounts of data. The number of interaction points where data is collected is increasing and the amount of data collected at each point is increasing. Collected data is being retained for longer periods of time resulting in continual database growth. Data processing in a business process takes a variety of forms, such as data warehousing, decision support software, analytical software, customer relationship management. Such data processing invariably involves transforming the data for use. [0006]Business processes are also increasingly going "real-time." This trend has an interesting side effect. As business processes become more dependent on near continuous refresh of data, they become less tolerant of transition periods. [0007]Refresh transition occurs when the data changes. Multiple, related sets of data from multiple sources must be refreshed in a consistent manner with respect to time. The more dependent business processes are on up-to-date data, the smaller the time windows for updates. Decreasing time windows in conjunction with increasing amounts of data presents a process execution scalability problem. [0008]Schema transition occurs when the type of data collected changes. Business processes and partnerships evolve and integrate in unpredictable ways. The more dependent business processes are on up-to-date data, the smaller the time windows for implementing change. That is, scalability is not limited by physical storage of data, but by applications to transform the data for business use. This presents a process development scalability problem, as well as an execution scalability problem. The challenge then is to lower the cost of development of data routing and transformation applications while at the same time, providing scaleable execution environments to respond to the ever increasing data flows and shrinking response time windows. [0009]Dataflow graphs are widely recognized as an effective means to specify data processing software. Their value lies in the succinct presentation and explicit definition of data transfers as the data progresses through a chain of transformation processes. Such dataflow graphs typically represent transformation processes as nodes having input and output ports, with the connections between nodes represented by arcs specifying sources and destinations for data transfer. The nodes may be hierarchical, with a single node at a high level representing a summary of a dataflow graph which can be decomposed into lower-level nodes and arcs, until primitive data transformations are presented at the lowest level of the hierarchy. The dataflow representation is found to be especially apt for multi-threaded execution environments such as parallel processing. [0010]With the wider availability of parallel processing, such as shared-memory multiprocessor (SMP) machines, clustered or distributed machines connected by networks, and single CPU machines executing multiple threads, the need for cost-effective and time-efficient programming methods for such execution environments is becoming increasingly important. The current state of the art in computer architecture design is shifting towards hyper-parallel computing. All the major CPU providers have embraced two trends, hyper-threading, and multiple core chips. [0011]Hyper-threading is the ability for a single CPU core to execute multiple threads simultaneously by interleaving them in the hardware's instruction pipeline. The typical CPU instruction pipeline has grown in length such that a single thread cannot keep the pipeline full. Interleaving instructions from multiple threads is the logical next step. [0012]Multiple core chips are the result of ever increasing chip real estate due to shrinking circuit size. It is equivalent to shrinking a multiple processor SMP server onto a single piece of silicon. For example, Sun Microsystems plans to have a single chip with 8 cores, with each core capable of executing 4 threads simultaneously. This is the equivalent of a 32-processor machine on 1 chip. This would enable a 64-processor machine to execute 64*32=2048 threads in parallel. Server hardware performance is set to expand rapidly for those applications that can take advantage of hyper-parallel computing. [0013]As used herein, "multi-threading" is intended to include multiple core architectures, i.e. a distinction is not made between parallel processing architectures such as SMP machines or a single CPU machine executing multiple threads. The current invention is applicable to all parallel processing architectures, e.g. a "thread" might be a process on a CPU in a multi-core SMP machine. [0014]The future of data integration will require both scalability in process execution and also scalability in process development. Parallel processing is a primary approach to execution scalability yet it typically increases the complexity of development. The paradox arises from the requirement of developing robust, complex, parallel applications in ever diminishing time frames. [0015]Since they are found to be effective, dataflow graphs have been used for both the specification and design of computer software as well as for documentation, user application training, and supporting code maintenance or modification activities. Further attempts have been made to use dataflow graphs as the basis for code synthesis. The goal has been to design the software using the dataflow graph representation and then use the resulting graphs to synthesize code for execution by associating software library functions in imperative languages or objects in declarative languages with the nodes of the dataflow graph. Difficulties encountered with prior implementations are limited flexibility/expressive power in component linking such that 1) Not all repeating dataflow patterns can be encapsulated in reusable components such that end users quite often have to "reinvent" those patterns in each application. 2) Sub-partitioning hierarchical dataflows becomes prohibitively expensive when attempting to utilize alternative dimensions of parallelism. [0016]The result has been that while dataflow graphs are widely used for system specification and design, and attempts have been made to synthesize code from such dataflow graphs, the two goals of process development scalability and process execution scalability have yet to be simultaneously achieved. [0017]Previous attempts to synthesize code directly from dataflow graphs achieve execution scalability but do so only in limited cases where the dimensions of parallelism exploited match well with the limited degrees of parallelism exposed. Many real world cases are excluded due to the limited flexibility/expressive power in component linking thus impacting reuse and ultimately development scalability. [0018]Alternatively the production code is sometimes written in a separate process from the dataflow design stage. Such an approach is acceptable if the pace of business process change is slow enough to allow high-performance production code to be written, by hand, after the system design is complete. [0019]There exists, however, a significant and growing class of data intensive high-performance applications where both approaches above are unacceptable. That is, there is a significant class of applications for which the delay between requirements change and working high-performance implementation must be minimized. These are the applications that are based on the growing flood of real-time data. When schema transition of real-time data occurs, the business processes dependent on that data cannot go off-line. New implementations, based on the new schema, must be available. The development of high-performance production code must not become the bottleneck in real-time business process change. In these cases, both the cost and time for creation of the code and its execution time must be held to a minimum. To minimize the cost and time of code creation, a generic hierarchical dataflow representation of the system must be retained at design time. This representation must be then be automatically transformed into a parallel, type-specific, non-hierarchical representation for efficient execution. [0020]An example of a dataflow graph development system is found in U.S. Pat. No. 5,999,729. An example of a deadlock resolution system in a multi-threaded environment is found in U.S. Pat. No. 6,088,716. Deadlock detection and correction in process networks are known, see, R. Stevens, M. Wan, P. Laramie, T. Parks & E. Lee, Implementation of Process Networks in Java, http://www.ait.nrl.navy.mil/pgmt/PNpaper.pdf, July 1997. An example of a parallel programming environment is found in U.S. Pat. No. 6,311,265. All references cited herein are incorporated by reference. [0021]It would therefore be a significant advantage if the cost-effectiveness of the graphical dataflow representation for design could be used to synthesize executable code with performance adequate for short-term production. SUMMARY OF THE INVENTION [0022]The problems with the synthesis of computationally efficient data transformation applications from dataflow graphs outlined above are addressed by the system and methods provided herein. The system and method herein provides for dataflow application development particularly suited for parallel architectures when the synthesis process is decomposed into a sequence of steps. The systems and methods hereof are particularly useful when data must be processed or examined quickly and efficiently and scalability in design as well as execution is important. Continue reading... Full patent description for Development environment for data transformation applications Brief Patent Description - Full Patent Description - Patent Application Claims Click on the above for other options relating to this Development environment for data transformation applications patent application. Patent Applications in related categories: 20080244539 - Enabling analysis of software source code - A source code clarification system is described. In various embodiments, the source code clarification system receives clarified source code and transforms the clarified source code into standard source code or object code that implements asynchronous components. The standard software source code can contain expressions for enabling asynchronous communications. The clarified ... ###  How KEYWORD MONITOR works... a FREE service from FreshPatents1. Sign up (takes 30 seconds). 2. Fill in the keywords to be monitored. 3. Each week you receive an email with patent applications related to your keywords. Start now! - Receive info on patent apps like Development environment for data transformation applications or other areas of interest. ### Previous Patent Application: Compiler method for eliminating redundant read-modify-write code sequences in non-vectorizable code Next Patent Application: Framework for parallelizing general reduction Industry Class: Data processing: software development, installation, and management ### FreshPatents.com Support Thank you for viewing the Development environment for data transformation applications patent info. IP-related news and info Results in 2.72704 seconds Other interesting Feshpatents.com categories: Daimler Chrysler , DirecTV , Exxonmobil Chemical Company , Goodyear , Intel , Kyocera Wireless , |
||
