Search results for: Markov dynamical system
Commenced in January 2007
Frequency: Monthly
Edition: International
Paper Count: 8488

Search results for: Markov dynamical system

8488 On Adaptive Optimization of Filter Performance Based on Markov Representation for Output Prediction Error

Authors: Hong Son Hoang, Remy Baraille

Abstract:

This paper addresses the problem of how one can improve the performance of a non-optimal filter. First the theoretical question on dynamical representation for a given time correlated random process is studied. It will be demonstrated that for a wide class of random processes, having a canonical form, there exists a dynamical system equivalent in the sense that its output has the same covariance function. It is shown that the dynamical approach is more effective for simulating and estimating a Markov and non- Markovian random processes, computationally is less demanding, especially with increasing of the dimension of simulated processes. Numerical examples and estimation problems in low dimensional systems are given to illustrate the advantages of the approach. A very useful application of the proposed approach is shown for the problem of state estimation in very high dimensional systems. Here a modified filter for data assimilation in an oceanic numerical model is presented which is proved to be very efficient due to introducing a simple Markovian structure for the output prediction error process and adaptive tuning some parameters of the Markov equation.

Keywords: Statistical simulation, canonical form, dynamical system, Markov and non-Markovian processes, data assimilation.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1250
8487 On Diffusion Approximation of Discrete Markov Dynamical Systems

Authors: Jevgenijs Carkovs

Abstract:

The paper is devoted to stochastic analysis of finite dimensional difference equation with dependent on ergodic Markov chain increments, which are proportional to small parameter ". A point-form solution of this difference equation may be represented as vertexes of a time-dependent continuous broken line given on the segment [0,1] with "-dependent scaling of intervals between vertexes. Tending " to zero one may apply stochastic averaging and diffusion approximation procedures and construct continuous approximation of the initial stochastic iterations as an ordinary or stochastic Ito differential equation. The paper proves that for sufficiently small " these equations may be successfully applied not only to approximate finite number of iterations but also for asymptotic analysis of iterations, when number of iterations tends to infinity.

Keywords: Markov dynamical system, diffusion approximation, equilibrium stochastic stability.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1515
8486 Evolutionary Training of Hybrid Systems of Recurrent Neural Networks and Hidden Markov Models

Authors: Rohitash Chandra, Christian W. Omlin

Abstract:

We present a hybrid architecture of recurrent neural networks (RNNs) inspired by hidden Markov models (HMMs). We train the hybrid architecture using genetic algorithms to learn and represent dynamical systems. We train the hybrid architecture on a set of deterministic finite-state automata strings and observe the generalization performance of the hybrid architecture when presented with a new set of strings which were not present in the training data set. In this way, we show that the hybrid system of HMM and RNN can learn and represent deterministic finite-state automata. We ran experiments with different sets of population sizes in the genetic algorithm; we also ran experiments to find out which weight initializations were best for training the hybrid architecture. The results show that the hybrid architecture of recurrent neural networks inspired by hidden Markov models can train and represent dynamical systems. The best training and generalization performance is achieved when the hybrid architecture is initialized with random real weight values of range -15 to 15.

Keywords: Deterministic finite-state automata, genetic algorithm, hidden Markov models, hybrid systems and recurrent neural networks.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1841
8485 Mathematical Approach towards Fault Detection and Isolation of Linear Dynamical Systems

Authors: V.Manikandan, N.Devarajan

Abstract:

The main objective of this work is to provide a fault detection and isolation based on Markov parameters for residual generation and a neural network for fault classification. The diagnostic approach is accomplished in two steps: In step 1, the system is identified using a series of input / output variables through an identification algorithm. In step 2, the fault is diagnosed comparing the Markov parameters of faulty and non faulty systems. The Artificial Neural Network is trained using predetermined faulty conditions serves to classify the unknown fault. In step 1, the identification is done by first formulating a Hankel matrix out of Input/ output variables and then decomposing the matrix via singular value decomposition technique. For identifying the system online sliding window approach is adopted wherein an open slit slides over a subset of 'n' input/output variables. The faults are introduced at arbitrary instances and the identification is carried out in online. Fault residues are extracted making a comparison of the first five Markov parameters of faulty and non faulty systems. The proposed diagnostic approach is illustrated on benchmark problems with encouraging results.

Keywords: Artificial neural network, Fault Diagnosis, Identification, Markov parameters.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1582
8484 Model-Free Distributed Control of Dynamical Systems

Authors: Javad Khazaei, Rick S. Blum

Abstract:

Distributed control is an efficient and flexible approach for coordination of multi-agent systems. One of the main challenges in designing a distributed controller is identifying the governing dynamics of the dynamical systems. Data-driven system identification is currently undergoing a revolution. With the availability of high-fidelity measurements and historical data, model-free identification of dynamical systems can facilitate the control design without tedious modeling of high-dimensional and/or nonlinear systems. This paper develops a distributed control design using consensus theory for linear and nonlinear dynamical systems using sparse identification of system dynamics. Compared with existing consensus designs that heavily rely on knowing the detailed system dynamics, the proposed model-free design can accurately capture the dynamics of the system with available measurements and input data and provide guaranteed performance in consensus and tracking problems. Heterogeneous damped oscillators are chosen as examples of dynamical system for validation purposes.

Keywords: Consensus tracking, distributed control, model-free control, sparse identification of dynamical systems.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 438
8483 An Optimal Bayesian Maintenance Policy for a Partially Observable System Subject to Two Failure Modes

Authors: Akram Khaleghei Ghosheh Balagh, Viliam Makis, Leila Jafari

Abstract:

In this paper, we present a new maintenance model for a partially observable system subject to two failure modes, namely a catastrophic failure and a failure due to the system degradation. The system is subject to condition monitoring and the degradation process is described by a hidden Markov model. A cost-optimal Bayesian control policy is developed for maintaining the system. The control problem is formulated in the semi-Markov decision process framework. An effective computational algorithm is developed, illustrated by a numerical example.

Keywords: Partially observable system, hidden Markov model, competing risks, multivariate Bayesian control.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 2139
8482 Investigation on Performance of Change Point Algorithm in Time Series Dynamical Regimes and Effect of Data Characteristics

Authors: Farhad Asadi, Mohammad Javad Mollakazemi

Abstract:

In this paper, Bayesian online inference in models of data series are constructed by change-points algorithm, which separated the observed time series into independent series and study the change and variation of the regime of the data with related statistical characteristics. variation of statistical characteristics of time series data often represent separated phenomena in the some dynamical system, like a change in state of brain dynamical reflected in EEG signal data measurement or a change in important regime of data in many dynamical system. In this paper, prediction algorithm for studying change point location in some time series data is simulated. It is verified that pattern of proposed distribution of data has important factor on simpler and smother fluctuation of hazard rate parameter and also for better identification of change point locations. Finally, the conditions of how the time series distribution effect on factors in this approach are explained and validated with different time series databases for some dynamical system.

Keywords: Time series, fluctuation in statistical characteristics, optimal learning.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1753
8481 A New Heuristic Statistical Methodology for Optimizing Queuing Networks Using Discreet Event Simulation

Authors: Mohamad Mahdavi

Abstract:

Most of the real queuing systems include special properties and constraints, which can not be analyzed directly by using the results of solved classical queuing models. Lack of Markov chains features, unexponential patterns and service constraints, are the mentioned conditions. This paper represents an applied general algorithm for analysis and optimizing the queuing systems. The algorithm stages are described through a real case study. It is consisted of an almost completed non-Markov system with limited number of customers and capacities as well as lots of common exception of real queuing networks. Simulation is used for optimizing this system. So introduced stages over the following article include primary modeling, determining queuing system kinds, index defining, statistical analysis and goodness of fit test, validation of model and optimizing methods of system with simulation.

Keywords: Estimation, queuing system, simulation model, probability distribution, non-Markov chain.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1569
8480 A Simplified Higher-Order Markov Chain Model

Authors: Chao Wang, Ting-Zhu Huang, Chen Jia

Abstract:

In this paper, we present a simplified higher-order Markov chain model for multiple categorical data sequences also called as simplified higher-order multivariate Markov chain model.

Keywords: Higher-order multivariate Markov chain model, Categorical data sequences, Multivariate Markov chain.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 3225
8479 Systems with Queueing and their Simulation

Authors: Miloš Šeda, Pavel Ošmera, Jindřich Petrucha

Abstract:

In the queueing theory, it is assumed that customer arrivals correspond to a Poisson process and service time has the exponential distribution. Using these assumptions, the behaviour of the queueing system can be described by means of Markov chains and it is possible to derive the characteristics of the system. In the paper, these theoretical approaches are presented on several types of systems and it is also shown how to compute the characteristics in a situation when these assumptions are not satisfied

Keywords: Queueing theory, Poisson process, Markov chains.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1233
8478 Regularization of the Trajectories of Dynamical Systems by Adjusting Parameters

Authors: Helle Hein, Ülo Lepik

Abstract:

A gradient learning method to regulate the trajectories of some nonlinear chaotic systems is proposed. The method is motivated by the gradient descent learning algorithms for neural networks. It is based on two systems: dynamic optimization system and system for finding sensitivities. Numerical results of several examples are presented, which convincingly illustrate the efficiency of the method.

Keywords: Chaos, Dynamical Systems, Learning, Neural Networks

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1323
8477 Residual Life Prediction for a System Subject to Condition Monitoring and Two Failure Modes

Authors: Akram Khaleghei Ghosheh Balagh, Viliam Makis

Abstract:

In this paper, we investigate the residual life prediction problem for a partially observable system subject to two failure modes, namely a catastrophic failure and a failure due to the system degradation. The system is subject to condition monitoring and the degradation process is described by a hidden Markov model with unknown parameters. The parameter estimation procedure based on an EM algorithm is developed and the formulas for the conditional reliability function and the mean residual life are derived, illustrated by a numerical example.

Keywords: Partially observable system, hidden Markov model, competing risks, residual life prediction.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1991
8476 Word Recognition and Learning based on Associative Memories and Hidden Markov Models

Authors: Zöhre Kara Kayikci, Günther Palm

Abstract:

A word recognition architecture based on a network of neural associative memories and hidden Markov models has been developed. The input stream, composed of subword-units like wordinternal triphones consisting of diphones and triphones, is provided to the network of neural associative memories by hidden Markov models. The word recognition network derives words from this input stream. The architecture has the ability to handle ambiguities on subword-unit level and is also able to add new words to the vocabulary during performance. The architecture is implemented to perform the word recognition task in a language processing system for understanding simple command sentences like “bot show apple".

Keywords: Hebbian learning, hidden Markov models, neuralassociative memories, word recognition.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1464
8475 Dynamical Transmission Model of Chikungunya in Thailand

Authors: P. Pongsumpun

Abstract:

One of the important tropical diseases is Chikunkunya. This disease is transmitted between the human by the insect-borne virus, of the genus Alphavirus. It occurs in Africa, Asia and the Indian subcontinent. In Thailand, the incidences due to this disease are increasing every year. In this study, the transmission of this disease is studied through dynamical model analysis.

Keywords: Chikunkunya, dynamical model, Endemic region, Routh-Hurwitz criteria.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1314
8474 Control of Chaotic Dynamical Systems using RBF Networks

Authors: Yoichi Ishikawa, Yuichi Masukake, Yoshihisa Ishida

Abstract:

This paper presents a novel control method based on radial basis function networks (RBFNs) for chaotic dynamical systems. The proposed method first identifies the nonlinear part of the chaotic system off-line and then constructs a model-following controller using only the estimated system parameters. Simulation results show the effectiveness of the proposed control scheme.

Keywords: Chaos, nonlinear plant, radial basis function network.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1592
8473 Designing Intelligent Adaptive Controller for Nonlinear Pendulum Dynamical System

Authors: R. Ghasemi, M. R. Rahimi Khoygani

Abstract:

This paper proposes the designing direct adaptive neural controller to apply for a class of a nonlinear pendulum dynamic system. The radial basis function (RBF) neural adaptive controller is robust in presence of external and internal uncertainties. Both the effectiveness of the controller and robustness against disturbances are importance of this paper. The simulation results show the promising performance of the proposed controller.

Keywords: Adaptive Neural Controller, Nonlinear Dynamical, Neural Network, RBF, Driven Pendulum, Position Control.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 2537
8472 Dynamical Network Transmission of H1N1 Virus at the Local Level Transmission Model

Authors: P. Pongsumpun

Abstract:

A new strain of Type A influenza virus can cause the transmission of H1N1 virus. This virus can spread between the people by coughing and sneezing. Because the people are always movement, so this virus can be easily spread. In this study, we construct the dynamical network model of H1N1 virus by separating the human into five groups; susceptible, exposed, infectious, quarantine and recovered groups. The movement of people between houses (local level) is considered. The behaviors of solutions to our dynamical model are shown for the different parameters.

Keywords: Dynamical network, H1N1virus, local level, simulation.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1494
8471 Javanese Character Recognition Using Hidden Markov Model

Authors: Anastasia Rita Widiarti, Phalita Nari Wastu

Abstract:

Hidden Markov Model (HMM) is a stochastic method which has been used in various signal processing and character recognition. This study proposes to use HMM to recognize Javanese characters from a number of different handwritings, whereby HMM is used to optimize the number of state and feature extraction. An 85.7 % accuracy is obtained as the best result in 16-stated vertical model using pure HMM. This initial result is satisfactory for prompting further research.

Keywords: Character recognition, off-line handwritingrecognition, Hidden Markov Model.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1937
8470 A Hybrid System of Hidden Markov Models and Recurrent Neural Networks for Learning Deterministic Finite State Automata

Authors: Pavan K. Rallabandi, Kailash C. Patidar

Abstract:

In this paper, we present an optimization technique or a learning algorithm using the hybrid architecture by combining the most popular sequence recognition models such as Recurrent Neural Networks (RNNs) and Hidden Markov models (HMMs). In order to improve the sequence/pattern recognition/classification performance by applying a hybrid/neural symbolic approach, a gradient descent learning algorithm is developed using the Real Time Recurrent Learning of Recurrent Neural Network for processing the knowledge represented in trained Hidden Markov Models. The developed hybrid algorithm is implemented on automata theory as a sample test beds and the performance of the designed algorithm is demonstrated and evaluated on learning the deterministic finite state automata.

Keywords: Hybrid systems, Hidden Markov Models, Recurrent neural networks, Deterministic finite state automata.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 2829
8469 Musical Instrument Classification Using Embedded Hidden Markov Models

Authors: Ehsan Amid, Sina Rezaei Aghdam

Abstract:

In this paper, a novel method for recognition of musical instruments in a polyphonic music is presented by using an embedded hidden Markov model (EHMM). EHMM is a doubly embedded HMM structure where each state of the external HMM is an independent HMM. The classification is accomplished for two different internal HMM structures where GMMs are used as likelihood estimators for the internal HMMs. The results are compared to those achieved by an artificial neural network with two hidden layers. Appropriate classification accuracies were achieved both for solo instrument performance and instrument combinations which demonstrates that the new approach outperforms the similar classification methods by means of the dynamic of the signal.

Keywords: hidden Markov model (HMM), embedded hidden Markov models (EHMM), MFCC, musical instrument.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1836
8468 Maintenance Alternatives Related to Costs of Wind Turbines Using Finite State Markov Model

Authors: Boukelkoul Lahcen

Abstract:

The cumulative costs for O&M may represent as much as 65%-90% of the turbine's investment cost. Nowadays the cost effectiveness concept becomes a decision-making and technology evaluation metric. The cost of energy metric accounts for the effect replacement cost and unscheduled maintenance cost parameters. One key of the proposed approach is the idea of maintaining the WTs which can be captured via use of a finite state Markov chain. Such a model can be embedded within a probabilistic operation and maintenance simulation reflecting the action to be done. In this paper, an approach of estimating the cost of O&M is presented. The finite state Markov model is used for decision problems with number of determined periods (life cycle) to predict the cost according to various options of maintenance.

Keywords: Cost, finite state, Markov model, operation, maintenance.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1425
8467 A Novel Convergence Accelerator for the LMS Adaptive Algorithm

Authors: Jeng-Shin Sheu, Jenn-Kaie Lain, Tai-Kuo Woo, Jyh-Horng Wen

Abstract:

The least mean square (LMS) algorithmis one of the most well-known algorithms for mobile communication systems due to its implementation simplicity. However, the main limitation is its relatively slow convergence rate. In this paper, a booster using the concept of Markov chains is proposed to speed up the convergence rate of LMS algorithms. The nature of Markov chains makes it possible to exploit the past information in the updating process. Moreover, since the transition matrix has a smaller variance than that of the weight itself by the central limit theorem, the weight transition matrix converges faster than the weight itself. Accordingly, the proposed Markov-chain based booster thus has the ability to track variations in signal characteristics, and meanwhile, it can accelerate the rate of convergence for LMS algorithms. Simulation results show that the LMS algorithm can effectively increase the convergence rate and meantime further approach the Wiener solution, if the Markov-chain based booster is applied. The mean square error is also remarkably reduced, while the convergence rate is improved.

Keywords: LMS, Markov chain, convergence rate, accelerator.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1710
8466 The First Integral Approach in Stability Problem of Large Scale Nonlinear Dynamical Systems

Authors: M. Kidouche, H. Habbi, M. Zelmat, S. Grouni

Abstract:

In analyzing large scale nonlinear dynamical systems, it is often desirable to treat the overall system as a collection of interconnected subsystems. Solutions properties of the large scale system are then deduced from the solution properties of the individual subsystems and the nature of the interconnections. In this paper a new approach is proposed for the stability analysis of large scale systems, which is based upon the concept of vector Lyapunov functions and the decomposition methods. The present results make use of graph theoretic decomposition techniques in which the overall system is partitioned into a hierarchy of strongly connected components. We show then, that under very reasonable assumptions, the overall system is stable once the strongly connected subsystems are stables. Finally an example is given to illustrate the constructive methodology proposed.

Keywords: Comparison principle, First integral, Large scale system, Lyapunov stability.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1482
8465 A Hidden Markov Model for Modeling Pavement Deterioration under Incomplete Monitoring Data

Authors: Nam Lethanh, Bryan T. Adey

Abstract:

In this paper, the potential use of an exponential hidden Markov model to model a hidden pavement deterioration process, i.e. one that is not directly measurable, is investigated. It is assumed that the evolution of the physical condition, which is the hidden process, and the evolution of the values of pavement distress indicators, can be adequately described using discrete condition states and modeled as a Markov processes. It is also assumed that condition data can be collected by visual inspections over time and represented continuously using an exponential distribution. The advantage of using such a model in decision making process is illustrated through an empirical study using real world data.

Keywords: Deterioration modeling, Exponential distribution, Hidden Markov model, Pavement management

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 2255
8464 Application of Smooth Ergodic Hidden Markov Model in Text to Speech Systems

Authors: Armin Ghayoori, Faramarz Hendessi, Asrar Sheikh

Abstract:

In developing a text-to-speech system, it is well known that the accuracy of information extracted from a text is crucial to produce high quality synthesized speech. In this paper, a new scheme for converting text into its equivalent phonetic spelling is introduced and developed. This method is applicable to many applications in text to speech converting systems and has many advantages over other methods. The proposed method can also complement the other methods with a purpose of improving their performance. The proposed method is a probabilistic model and is based on Smooth Ergodic Hidden Markov Model. This model can be considered as an extension to HMM. The proposed method is applied to Persian language and its accuracy in converting text to speech phonetics is evaluated using simulations.

Keywords: Hidden Markov Models, text, synthesis.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1503
8463 Availability Analysis of Milling System in a Rice Milling Plant

Authors: P. C. Tewari, Parveen Kumar

Abstract:

The paper describes the availability analysis of milling system of a rice milling plant using probabilistic approach. The subsystems under study are special purpose machines. The availability analysis of the system is carried out to determine the effect of failure and repair rates of each subsystem on overall performance (i.e. steady state availability) of system concerned. Further, on the basis of effect of repair rates on the system availability, maintenance repair priorities have been suggested. The problem is formulated using Markov Birth-Death process taking exponential distribution for probable failures and repair rates. The first order differential equations associated with transition diagram are developed by using mnemonic rule. These equations are solved using normalizing conditions and recursive method to drive out the steady state availability expression of the system. The findings of the paper are presented and discussed with the plant personnel to adopt a suitable maintenance policy to increase the productivity of the rice milling plant.

Keywords: Markov process, milling system, availability modeling, rice milling plant.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1510
8462 The Effect of Measurement Distribution on System Identification and Detection of Behavior of Nonlinearities of Data

Authors: Mohammad Javad Mollakazemi, Farhad Asadi, Aref Ghafouri

Abstract:

In this paper, we considered and applied parametric modeling for some experimental data of dynamical system. In this study, we investigated the different distribution of output measurement from some dynamical systems. Also, with variance processing in experimental data we obtained the region of nonlinearity in experimental data and then identification of output section is applied in different situation and data distribution. Finally, the effect of the spanning the measurement such as variance to identification and limitation of this approach is explained.

Keywords: Gaussian process, Nonlinearity distribution, Particle filter.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1657
8461 An Inverse Optimal Control Approach for the Nonlinear System Design Using ANN

Authors: M. P. Nanda Kumar, K. Dheeraj

Abstract:

The design of a feedback controller, so as to minimize a given performance criterion, for a general non-linear dynamical system is difficult; if not impossible. But for a large class of non-linear dynamical systems, the open loop control that minimizes a performance criterion can be obtained using calculus of variations and Pontryagin’s minimum principle. In this paper, the open loop optimal trajectories, that minimizes a given performance measure, is used to train the neural network whose inputs are state variables of non-linear dynamical systems and the open loop optimal control as the desired output. This trained neural network is used as the feedback controller. In other words, attempts are made here to solve the “inverse optimal control problem” by using the state and control trajectories that are optimal in an open loop sense.

Keywords: Inverse Optimal Control, Radial basis function neural network, Controller Design.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 2244
8460 Markov Game Controller Design Algorithms

Authors: Rajneesh Sharma, M. Gopal

Abstract:

Markov games are a generalization of Markov decision process to a multi-agent setting. Two-player zero-sum Markov game framework offers an effective platform for designing robust controllers. This paper presents two novel controller design algorithms that use ideas from game-theory literature to produce reliable controllers that are able to maintain performance in presence of noise and parameter variations. A more widely used approach for controller design is the H∞ optimal control, which suffers from high computational demand and at times, may be infeasible. Our approach generates an optimal control policy for the agent (controller) via a simple Linear Program enabling the controller to learn about the unknown environment. The controller is facing an unknown environment, and in our formulation this environment corresponds to the behavior rules of the noise modeled as the opponent. Proposed controller architectures attempt to improve controller reliability by a gradual mixing of algorithmic approaches drawn from the game theory literature and the Minimax-Q Markov game solution approach, in a reinforcement-learning framework. We test the proposed algorithms on a simulated Inverted Pendulum Swing-up task and compare its performance against standard Q learning.

Keywords: Reinforcement learning, Markov Decision Process, Matrix Games, Markov Games, Smooth Fictitious play, Controller, Inverted Pendulum.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1470
8459 Performance Modeling and Availability Analysis of Yarn Dyeing System of a Textile Industry

Authors: P. C. Tewari, Rajiv Kumar, Dinesh Khanduja

Abstract:

This paper discusses the performance modeling and availability analysis of Yarn Dyeing System of a Textile Industry. The Textile Industry is a complex and repairable engineering system. Yarn Dyeing System of Textile Industry consists of five subsystems arranged in series configuration. For performance modeling and analysis of availability, a performance evaluating model has been developed with the help of mathematical formulation based on Markov-Birth-Death Process. The differential equations have been developed on the basis of Probabilistic Approach using a Transition Diagram. These equations have further been solved using normalizing condition in order to develop the steady state availability, a performance measure of the system concerned. The system performance has been further analyzed with the help of decision matrices. These matrices provide various availability levels for different combinations of failure and repair rates for various subsystems. The findings of this paper are therefore, considered to be useful for the analysis of availability and determination of the best possible maintenance strategies which can be implemented in future to enhance the system performance.

Keywords: Availability Analysis, Markov Process, Performance Modeling, Steady State Availability.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 2251