Home About us Contact
|
Home About us Contact | ||
|
|
||
Learning Algorithms (learning + algorithms)
Kinds of Learning Algorithms Selected AbstractsHIGH-DIMENSIONAL LEARNING FRAMEWORK FOR ADAPTIVE DOCUMENT FILTERING,COMPUTATIONAL INTELLIGENCE, Issue 1 2003Wai Lam We investigate the unique requirements of the adaptive textual document filtering problem and propose a new high-dimensional on-line learning framework, known as the REPGER (relevant feature pool with good training example retrieval rule) algorithm to tackle this problem. Our algorithm possesses three characteristics. First, it maintains a pool of selective features with potentially high predictive power to predict document relevance. Second, besides retrieving documents according to their predicted relevance, it also retrieves incoming documents that are considered good training examples. Third, it can dynamically adjust the dissemination threshold throughout the filtering process so as to maintain a good filtering performance in a fully interactive environment. We have conducted experiments on three document corpora, namely, Associated Press, Foreign Broadcast Information Service, and Wall Street Journal to compare the performance of our REPGER algorithm with two existing on-line learning algorithms. The results demonstrate that our REPGER algorithm gives better performance most of the time. Comparison with the TREC (Text Retrieval Conference) adaptive text filtering track participants was also made. The result shows that our REPGER algorithm is comparable to them. [source] An early warning system for detection of financial crisis using financial market volatilityEXPERT SYSTEMS, Issue 2 2006Kyong Joo Oh Abstract: This study proposes an early warning system (EWS) for detection of financial crisis with a daily financial condition indicator (DFCI) designed to monitor the financial markets and provide warning signals. The proposed EWS differs from other commonly used EWSs in two aspects: (i) it is based on dynamic daily movements of the financial markets; and (ii) it is established as a pattern classifier, which identifies predefined unstable states in terms of financial market volatility. Indeed it issues warning signals on a daily basis by judging whether the financial market has entered a predefined unstable state or not. The major strength of a DFCI is that it can issue timely warning signals while other conventional EWSs must wait for the next round input of monthly or quarterly information. Construction of a DFCI consists of two steps where machine learning algorithms are expected to play a significant role, i.e. (i) establishing sub-DFCIs on various daily financial variables by an artificial neural network, and (ii) integrating the sub-DFCIs into an integrated DFCI by a genetic algorithm. The DFCI for the Korean financial market is built as an empirical case study. [source] Inducing safer oblique trees without costsEXPERT SYSTEMS, Issue 4 2005Sunil Vadera Abstract: Decision tree induction has been widely studied and applied. In safety applications, such as determining whether a chemical process is safe or whether a person has a medical condition, the cost of misclassification in one of the classes is significantly higher than in the other class. Several authors have tackled this problem by developing cost-sensitive decision tree learning algorithms or have suggested ways of changing the distribution of training examples to bias the decision tree learning process so as to take account of costs. A prerequisite for applying such algorithms is the availability of costs of misclassification. Although this may be possible for some applications, obtaining reasonable estimates of costs of misclassification is not easy in the area of safety. This paper presents a new algorithm for applications where the cost of misclassifications cannot be quantified, although the cost of misclassification in one class is known to be significantly higher than in another class. The algorithm utilizes linear discriminant analysis to identify oblique relationships between continuous attributes and then carries out an appropriate modification to ensure that the resulting tree errs on the side of safety. The algorithm is evaluated with respect to one of the best known cost-sensitive algorithms (ICET), a well-known oblique decision tree algorithm (OC1) and an algorithm that utilizes robust linear programming. [source] A levenberg,marquardt learning applied for recurrent neural identification and control of a wastewater treatment bioprocessINTERNATIONAL JOURNAL OF INTELLIGENT SYSTEMS, Issue 11 2009Ieroham S. Baruch The paper proposed a new recurrent neural network (RNN) model for systems identification and states estimation of nonlinear plants. The proposed RNN identifier is implemented in direct and indirect adaptive control schemes, incorporating a noise rejecting plant output filter and recurrent neural or linear-sliding mode controllers. For sake of comparison, the RNN model is learned both by the backpropagation and by the recursive Levenberg,Marquardt (L,M) learning algorithm. The estimated states and parameters of the RNN model are used for direct and indirect adaptive trajectory tracking control. The proposed direct and indirect schemes are applied for real-time control of wastewater treatment bioprocess, where a good, convergence, noise filtering, and low mean squared error of reference tracking is achieved for both learning algorithms, with priority of the L,M one. © 2009 Wiley Periodicals, Inc. [source] Sharing in teams of heterogeneous, collaborative learning agentsINTERNATIONAL JOURNAL OF INTELLIGENT SYSTEMS, Issue 2 2009Christopher M. Gifford This paper is focused on the effects of sharing knowledge and collaboration of multiple heterogeneous, intelligent agents (hardware or software) which work together to learn a task. As each agent employs a different machine learning technique, the system consists of multiple knowledge sources and their respective heterogeneous knowledge representations. Collaboration between agents involves sharing knowledge to both speed up team learning, as well as refine the team's overall performance and group behavior. Experiments have been performed that vary the team composition in terms of machine learning algorithms, learning strategies employed by the agents, and sharing frequency for a predator-prey cooperative pursuit task. For lifelong learning, heterogeneous learning teams were more successful than homogeneous learning counterparts. Interestingly, sharing increased the learning rate, but sharing with higher frequency showed diminishing results. Lastly, knowledge conflicts are reduced over time the more sharing takes place. These results support further investigation of the merits of heterogeneous learning. © 2008 Wiley Periodicals, Inc. [source] An introduction of the condition class space with continuous value discretization and rough set theoryINTERNATIONAL JOURNAL OF INTELLIGENT SYSTEMS, Issue 2 2006Malcolm J. Beynon The granularity of an information system has an incumbent effect on the efficacy of the analysis from many machine learning algorithms. An information system contains a universe of objects characterized and categorized by condition and decision attributes. To manage the concomitant granularity, a level of continuous value discretization (CVD) is often undertaken. In the case of the rough set theory (RST) methodology for object classification, the granularity contributes to the grouping of objects into condition classes with the same condition attribute values. This article exposits the effect of a level of CVD on the subsequent condition classes constructed, with the introduction of the condition class space,the domain within which the condition classes exist. This domain elucidates the association of the condition classes to the related decision outcomes,reflecting the inexactness incumbent when a level of CVD is undertaken. A series of measures is defined that quantify this association. Throughout this study and without loss of generality, the findings are made through the RST methodology. This further offers a novel exposition of the relationship between all the condition attributes and the RST-related reducts (subsets of condition attributes). © 2006 Wiley Periodicals, Inc. Int J Int Syst 21: 173,191, 2006. [source] Evolutionary learning of dynamic probabilistic models with large time lagsINTERNATIONAL JOURNAL OF INTELLIGENT SYSTEMS, Issue 5 2001Allan Tucker In this paper, we explore the automatic explanation of multivariate time series (MTS) through learning dynamic Bayesian networks (DBNs). We have developed an evolutionary algorithm which exploits certain characteristics of MTS in order to generate good networks as quickly as possible. We compare this algorithm to other standard learning algorithms that have traditionally been used for static Bayesian networks but are adapted for DBNs in this paper. These are extensively tested on both synthetic and real-world MTS for various aspects of efficiency and accuracy. By proposing a simple representation scheme, an efficient learning methodology, and several useful heuristics, we have found that the proposed method is more efficient for learning DBNs from MTS with large time lags, especially in time-demanding situations. © 2001 John Wiley & Sons, Inc. [source] Sampled-data iterative learning control with well-defined relative degreeINTERNATIONAL JOURNAL OF ROBUST AND NONLINEAR CONTROL, Issue 8 2004Mingxuan Sun Abstract This paper addresses the problem of iterative learning control with well-defined relative degree. The solution is a family of sampled-data learning algorithms using lower-order differentiations of the tracking error with the order less than the relative degree. A unified convergence condition for the family of learning algorithms is derived and is proved to be independent of the highest order of the differentiations. In the presence of initial condition errors, the system output is ensured to converge to the desired trajectory with a specified error bound at each sampling instant. The bound will reduce to zero whenever the bound on initial condition errors tends to zero. Numerical examples are provided to illustrate the tracking performance of the proposed learning algorithms. Copyright © 2004 John Wiley & Sons, Ltd. [source] Multiple classifier integration for the prediction of protein structural classesJOURNAL OF COMPUTATIONAL CHEMISTRY, Issue 14 2009Lei Chen Abstract Supervised classifiers, such as artificial neural network, partition trees, and support vector machines, are often used for the prediction and analysis of biological data. However, choosing an appropriate classifier is not straightforward because each classifier has its own strengths and weaknesses, and each biological dataset has its own characteristics. By integrating many classifiers together, people can avoid the dilemma of choosing an individual classifier out of many to achieve an optimized classification results (Rahman et al., Multiple Classifier Combination for Character Recognition: Revisiting the Majority Voting System and Its Variation, Springer, Berlin, 2002, 167,178). The classification algorithms come from Weka (Witten and Frank, Data Mining: Practical Machine Learning Tools and Techniques, Morgan Kaufmann, San Francisco, 2005) (a collection of software tools for machine learning algorithms). By integrating many predictors (classifiers) together through simple voting, the correct prediction (classification) rates are 65.21% and 65.63% for a basic training dataset and an independent test set, respectively. These results are better than any single machine learning algorithm collected in Weka when exactly the same data are used. Furthermore, we introduce an integration strategy which takes care of both classifier weightings and classifier redundancy. A feature selection strategy, called minimum redundancy maximum relevance (mRMR), is transferred into algorithm selection to deal with classifier redundancy in this research, and the weightings are based on the performance of each classifier. The best classification results are obtained when 11 algorithms are selected by mRMR method, and integrated together through majority votes with weightings. As a result, the prediction correct rates are 68.56% and 69.29% for the basic training dataset and the independent test dataset, respectively. The web-server is available at http://chemdata.shu.edu.cn/protein_st/. © 2009 Wiley Periodicals, Inc. J Comput Chem, 2009 [source] Autonomous off-road navigation with end-to-end learning for the LAGR programJOURNAL OF FIELD ROBOTICS (FORMERLY JOURNAL OF ROBOTIC SYSTEMS), Issue 1 2009Max Bajracharya We describe a fully integrated real-time system for autonomous off-road navigation that uses end-to-end learning from onboard proprioceptive sensors, operator input, and stereo cameras to adapt to local terrain and extend terrain classification into the far field to avoid myopic behavior. The system consists of two learning algorithms: a short-range, geometry-based local terrain classifier that learns from very few proprioceptive examples and is robust in many off-road environments; and a long-range, image-based classifier that learns from geometry-based classification and continuously generalizes geometry to appearance, making it effective even in complex terrain and varying lighting conditions. In addition to presenting the learning algorithms, we describe the system architecture and results from the Learning Applied to Ground Robots (LAGR) program's field tests. © 2008 Wiley Periodicals, Inc. [source] Learning in a hierarchical control system: 4D/RCS in the DARPA LAGR programJOURNAL OF FIELD ROBOTICS (FORMERLY JOURNAL OF ROBOTIC SYSTEMS), Issue 11-12 2006Jim Albus The Defense Applied Research Projects Agency (DARPA) Learning Applied to Ground Vehicles (LAGR) program aims to develop algorithms for autonomous vehicle navigation that learn how to operate in complex terrain. Over many years, the National Institute of Standards and Technology (NIST) has developed a reference model control system architecture called 4D/RCS that has been applied to many kinds of robot control, including autonomous vehicle control. For the LAGR program, NIST has embedded learning into a 4D/RCS controller to enable the small robot used in the program to learn to navigate through a range of terrain types. The vehicle learns in several ways. These include learning by example, learning by experience, and learning how to optimize traversal. Learning takes place in the sensory processing, world modeling, and behavior generation parts of the control system. The 4D/RCS architecture is explained in the paper, its application to LAGR is described, and the learning algorithms are discussed. Results are shown of the performance of the NIST control system on independently-conducted tests. Further work on the system and its learning capabilities is discussed. © 2007 Wiley Periodicals, Inc. [source] Multi-objective learning control for robotic manipulatorJOURNAL OF FIELD ROBOTICS (FORMERLY JOURNAL OF ROBOTIC SYSTEMS), Issue 10 2004Khin Kyu Kyu Win Several types of learning controllers have been proposed in the literature to improve the tracking performance of robot manipulators. In most cases, the learning algorithms emphasize mainly on a single objective of learning a desired motion of the end-effector. In some applications, more than one objective may be specified at the same time. For example, a robot may be required to follow a desired trajectory (primary objective) and at the same time avoid an obstacle (secondary objective). Thus, multi-objective learning control can be more effective to realize the collision-free tasks. In this paper, a multi-objective learning control problem is formulated and solved. In the proposed learning control system, the primary objective is to track a desired end-effector's motion and several secondary objectives can be specified for the desired orientation and for obstacles avoidance. To avoid obstacles in the workspace, a new learning concept called "region learning control" is also proposed in this paper. The proposed learning controllers do not require the exact knowledge of robot kinematics and dynamics. Sufficient condition is presented to guarantee the convergence of the learning system. The proposed learning controllers are applied to a four-link planar redundant manipulator and simulation results are presented to illustrate the performance. © 2004 Wiley Periodicals, Inc. [source] Machine learning approaches for prediction of linear B-cell epitopes on proteinsJOURNAL OF MOLECULAR RECOGNITION, Issue 3 2006Johannes Söllner Abstract Identification and characterization of antigenic determinants on proteins has received considerable attention utilizing both, experimental as well as computational methods. For computational routines mostly structural as well as physicochemical parameters have been utilized for predicting the antigenic propensity of protein sites. However, the performance of computational routines has been low when compared to experimental alternatives. Here we describe the construction of machine learning based classifiers to enhance the prediction quality for identifying linear B-cell epitopes on proteins. Our approach combines several parameters previously associated with antigenicity, and includes novel parameters based on frequencies of amino acids and amino acid neighborhood propensities. We utilized machine learning algorithms for deriving antigenicity classification functions assigning antigenic propensities to each amino acid of a given protein sequence. We compared the prediction quality of the novel classifiers with respect to established routines for epitope scoring, and tested prediction accuracy on experimental data available for HIV proteins. The major finding is that machine learning classifiers clearly outperform the reference classification systems on the HIV epitope validation set. Copyright © 2006 John Wiley & Sons, Ltd. [source] Field-Scale Application of Three Types of Neural Networks to Predict Ground-Water Levels,JOURNAL OF THE AMERICAN WATER RESOURCES ASSOCIATION, Issue 5 2007Tirusew Asefa Abstract:, In this paper, a field-scale applicability of three forms of artificial neural network algorithms in forecasting short-term ground-water levels at specific control points is presented. These algorithms are the feed-forward back propagation (FFBP), radial basis networks (RBN), and generalized regression networks (GRN). Ground-water level predictions from these algorithms are in turn to be used in an Optimized Regional Operations Plan that prescribes scheduled wellfield production for the coming four weeks. These models are up against each other for their accuracy of ground-water level predictions on lead times ranging from a week to four weeks, ease of implementation, and execution times (mainly training time). In total, 208 networks of each of the three algorithms were developed for the study. It is shown that although learning algorithms have emerged as a viable solution at field scale much larger than previously studied, no single algorithm performs consistently better than others on all the criteria. On average, FFBP networks are 20 and 26%, respectively, more accurate than RBN and GRN in forecasting one week ahead water levels and this advantage drops to 5 and 9% accuracy in forecasting four weeks ahead water levels, whereas GRN posted a training time that is only 5% of the training time taken by that of FFBP networks. This may suggest that in field-scale applications one may have to trade between the type of algorithm to be used and the degree to which a given objective is honored. [source] Using support vector machines for automatic new topic identificationPROCEEDINGS OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE & TECHNOLOGY (ELECTRONIC), Issue 1 2007Seda Ozmutlu Recent studies on automatic new topic identification in Web search engine user sessions demonstrated that learning algorithms such as neural networks and regression have been fairly successful in automatic new topic identification. In this study, we investigate whether another learning algorithm, Support Vector Machines (SVM) are successful in terms of identifying topic shifts and continuations. Sample data logs from the Norwegian search engine FAST (currently owned by Overture) and Excite are used in this study. Findings of this study suggest that support vector machines' performance depends on the characteristics of the dataset it is applied on. [source] Discovering robust protein biomarkers for disease from relative expression reversals in 2-D DIGE data.PROTEINS: STRUCTURE, FUNCTION AND BIOINFORMATICS, Issue 8 2007Troy J. Anderson Abstract This study assesses the ability of a novel family of machine learning algorithms to identify changes in relative protein expression levels, measured using 2-D DIGE data, which support accurate class prediction. The analysis was done using a training set of 36 total cellular lysates comprised of six normal and three cancer biological replicates (the remaining are technical replicates) and a validation set of four normal and two cancer samples. Protein samples were separated by 2-D DIGE and expression was quantified using DeCyder-2D Differential Analysis Software. The relative expression reversal (RER) classifier correctly classified 9/9 training biological samples (p<0.022) as estimated using a modified version of leave one out cross validation and 6/6 validation samples. The classification rule involved comparison of expression levels for a single pair of protein spots, tropomyosin isoforms and ,-enolase, both of which have prior association as potential biomarkers in cancer. The data was also analyzed using algorithms similar to those found in the extended data analysis package of DeCyder software. We propose that by accounting for sources of within- and between-gel variation, RER classifiers applied to 2-D DIGE data provide a useful approach for identifying biomarkers that discriminate among protein samples of interest. [source] An alternative evaluation of FMEA: Fuzzy ART algorithmQUALITY AND RELIABILITY ENGINEERING INTERNATIONAL, Issue 6 2009en Ayd, n Keskin Abstract Failure Mode and Effects Analysis (FMEA) is a technique used in the manufacturing industry to improve production quality and productivity. It is a method that evaluates possible failures in the system, design, process or service. It aims to continuously improve and decrease these kinds of failure modes. Adaptive Resonance Theory (ART) is one of the learning algorithms without consultants, which are developed for clustering problems in artificial neural networks. In the FMEA method, every failure mode in the system is analyzed according to severity, occurrence and detection. Then, risk priority number (RPN) is acquired by multiplication of these three factors and the necessary failures are improved with respect to the determined threshold value. In addition, there exist many shortcomings of the traditional FMEA method, which affect its efficiency and thus limit its realization. To respond to these difficulties, this study introduces the method named Fuzzy Adaptive Resonance Theory (Fuzzy ART), one of the ART networks, to evaluate RPN in FMEA. Copyright © 2008 John Wiley & Sons, Ltd. [source] Modeling and analysis of disease and risk factors through learning Bayesian networks from observational dataQUALITY AND RELIABILITY ENGINEERING INTERNATIONAL, Issue 3 2008Jing Li Abstract This paper focuses on identification of the relationships between a disease and its potential risk factors using Bayesian networks in an epidemiologic study, with the emphasis on integrating medical domain knowledge and statistical data analysis. An integrated approach is developed to identify the risk factors associated with patients' occupational histories and is demonstrated using real-world data. This approach includes several steps. First, raw data are preprocessed into a format that is acceptable to the learning algorithms of Bayesian networks. Some important considerations are discussed to address the uniqueness of the data and the challenges of the learning. Second, a Bayesian network is learned from the preprocessed data set by integrating medical domain knowledge and generic learning algorithms. Third, the relationships revealed by the Bayesian network are used for risk factor analysis, including identification of a group of people who share certain common characteristics and have a relatively high probability of developing the disease, and prediction of a person's risk of developing the disease given information on his/her occupational history. Copyright © 2007 John Wiley & Sons, Ltd. [source] On-line learning for very large data setsAPPLIED STOCHASTIC MODELS IN BUSINESS AND INDUSTRY, Issue 2 2005Léon Bottou Abstract The design of very large learning systems presents many unsolved challenges. Consider, for instance, a system that ,watches' television for a few weeks and learns to enumerate the objects present in these images. Most current learning algorithms do not scale well enough to handle such massive quantities of data. Experience suggests that the stochastic learning algorithms are best suited to such tasks. This is at first surprising because stochastic learning algorithms optimize the training error rather slowly. Our paper reconsiders the convergence speed in terms of how fast a learning algorithm optimizes the testing error. This reformulation shows the superiority of the well designed stochastic learning algorithm. Copyright © 2005 John Wiley & Sons, Ltd. [source] Online identification of nonlinear multivariable processes using self-generating RBF neural networksASIAN JOURNAL OF CONTROL, Issue 5 2010Karim Salahshoor Abstract This paper addresses the problem of online model identification for multivariable processes with nonlinear and time-varying dynamic characteristics. For this purpose, two online multivariable identification approaches with self-organizing neural network model structures will be presented. The two adaptive radial basis function (RBF) neural networks are called as the growing and pruning radial basis function (GAP-RBF) and minimal resource allocation network (MRAN). The resulting identification algorithms start without a predefined model structure and the dynamic model is generated autonomously using the sequential input-output data pairs in real-time applications. The extended Kalman filter (EKF) learning algorithm has been extended for both of the adaptive RBF-based neural network approaches to estimate the free parameters of the identified multivariable model. The unscented Kalman filter (UKF) has been proposed as an alternative learning algorithm to enhance the accuracy and robustness of nonlinear multivariable processes in both the GAP-RBF and MRAN based approaches. In addition, this paper intends to study comparatively the general applicability of the particle filter (PF)-based approaches for the case of non-Gaussian noisy environments. For this purpose, the Unscented Particle Filter (UPF) is employed to be used as alternative to the EKF and UKF for online parameter estimation of self-generating RBF neural networks. The performance of the proposed online identification approaches is evaluated on a highly nonlinear time-varying multivariable non-isothermal continuous stirred tank reactor (CSTR) benchmark problem. Simulation results demonstrate the good performances of all identification approaches, especially the GAP-RBF approach incorporated with the UKF and UPF learning algorithms. Copyright © 2010 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society [source] USING LEAST SQUARE SVM FOR NONLINEAR SYSTEMS MODELING AND CONTROLASIAN JOURNAL OF CONTROL, Issue 2 2007Haoran Zhang ABSTRACT Support vector machine is a learning technique based on the structural risk minimization principle, and it is also a class of regression method with good generalization ability. The paper firstly introduces the mathematical model of regression least squares support vector machine (LSSVM), and designs incremental learning algorithms by the calculation formula of block matrix, then uses LSSVM to model nonlinear system, based on which to control nonlinear systems by model predictive method. Simulation experiments indicate that the proposed method provides satisfactory performance, and it achieves superior modeling performance to the conventional method based on neural networks, moreover it achieves well control performance. [source] | |||||||||||||||||||