Search results for: Markov Games

Authors: Rajneesh Sharma, M. Gopal

Abstract:

Markov games are a generalization of Markov decision process to a multi-agent setting. Two-player zero-sum Markov game framework offers an effective platform for designing robust controllers. This paper presents two novel controller design algorithms that use ideas from game-theory literature to produce reliable controllers that are able to maintain performance in presence of noise and parameter variations. A more widely used approach for controller design is the H∞ optimal control, which suffers from high computational demand and at times, may be infeasible. Our approach generates an optimal control policy for the agent (controller) via a simple Linear Program enabling the controller to learn about the unknown environment. The controller is facing an unknown environment, and in our formulation this environment corresponds to the behavior rules of the noise modeled as the opponent. Proposed controller architectures attempt to improve controller reliability by a gradual mixing of algorithmic approaches drawn from the game theory literature and the Minimax-Q Markov game solution approach, in a reinforcement-learning framework. We test the proposed algorithms on a simulated Inverted Pendulum Swing-up task and compare its performance against standard Q learning.

Keywords: Reinforcement learning, Markov Decision Process, Matrix Games, Markov Games, Smooth Fictitious play, Controller, Inverted Pendulum.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1520

Authors: R. Sharma, M. Gopal

Abstract:

Markov games can be effectively used to design controllers for nonlinear systems. The paper presents two novel controller design algorithms by incorporating ideas from gametheory literature that address safety and consistency issues of the 'learned' control strategy. A more widely used approach for controller design is the H∞ optimal control, which suffers from high computational demand and at times, may be infeasible. We generate an optimal control policy for the agent (controller) via a simple Linear Program enabling the controller to learn about the unknown environment. The controller is facing an unknown environment and in our formulation this environment corresponds to the behavior rules of the noise modeled as the opponent. Proposed approaches aim to achieve 'safe-consistent' and 'safe-universally consistent' controller behavior by hybridizing 'min-max', 'fictitious play' and 'cautious fictitious play' approaches drawn from game theory. We empirically evaluate the approaches on a simulated Inverted Pendulum swing-up task and compare its performance against standard Q learning.

Keywords: Fictitious Play, Cautious Fictitious Play, InvertedPendulum, Controller, Markov Games, Mobile Robot.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1430

Authors: Widodo Budiharto, Michael Yoseph Ricky, Ro'fah Nur Rachmawati

Abstract:

The rapid development of the BlackBerry games industry and its development goals were not just for entertainment, but also used for educational of students interactively. Unfortunately the development of adaptive educational games on BlackBerry in Indonesian language that interesting and entertaining for learning process is very limited. This paper shows the research of development of novel adaptive educational games for students who can adjust the difficulty level of games based on the ability of the user, so that it can motivate students to continue to play these games. We propose a method where these games can adjust the level of difficulty, based on the assessment of the results of previous problems using neural networks with three inputs in the form of percentage correct, the speed of answer and interest mode of games (animation / lessons) and 1 output. The experimental results are presented and show the adaptive games are running well on mobile devices based on BlackBerry platform

Keywords: Adaptive games, neural networks, mobile games, BlackBerry

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1845

Authors: Chao Wang, Ting-Zhu Huang, Chen Jia

Abstract:

In this paper, we present a simplified higher-order Markov chain model for multiple categorical data sequences also called as simplified higher-order multivariate Markov chain model.

Keywords: Higher-order multivariate Markov chain model, Categorical data sequences, Multivariate Markov chain.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 3287

Authors: Roman V. Yampolskiy

Abstract:

Games can be classified as games of skill, games of chance or otherwise be classified as mixed. This paper deals with the topic of scientifically classifying mixed games as more reliant on elements of chance or elements of skill and ways to scientifically measure the amount of skill involved. This is predominantly useful for classification of games as legal or illegal in deferent jurisdictions based on the local gaming laws. We propose a novel measure of skill to chance ratio called the Game Skill Measure (GSM) and utilize it to calculate the skill component of a popular variant of Poker.

Keywords: Chance, Game, Skill, Luck.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1462

Authors: Fan-Yong Meng

Abstract:

In this paper, we first introduce the model of games on augmenting systems with a coalition structure, which can be seen as an extension of games on augmenting systems. The core of games on augmenting systems with a coalition structure is defined, and an equivalent form is discussed. Meantime, the Shapley function for this type of games is given, and two axiomatic systems of the given Shapley function are researched. When the given games are quasi convex, the relationship between the core and the Shapley function is discussed, which does coincide as in classical case. Finally, a numerical example is given.

Keywords: Cooperative game, augmenting system, Shapley function, core.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1194

Authors: Fan-Yong Meng, Yan Wang

Abstract:

In this paper, we discuss the egalitarianism solution (ES) and center-of-gravity of the imputation-set value (CIV) for bicooperative games, which can be seen as the extensions of the solutions for traditional games given by Dutta and Ray [1] and Driessen and Funaki [2]. Furthermore, axiomatic systems for the given values are proposed. Finally, a numerical example is offered to illustrate the player ES and CTV.

Keywords: Bicooperative games, egalitarianism solution, center of- gravity of the imputation-set value.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1339

Authors: Pei-Shan Wei, Szu-Ying Lee, Hsi-Peng Lu, Jen-Chuen Tzou, Chien-I Weng

Abstract:

Mobile social games recently become extremely popular, spawning a whole new entertainment culture. However, mobile game players are fickle, quickly and easily picking up and abandoning games. This pilot study seeks to identify factors that influence users to discontinuing playing mobile social games. We identified three sacrifices which can prompt users to abandon games: monetary sacrifice, time sacrifice and privacy sacrifice. The results showed that monetary sacrifice has a greater impact than the other two factors in causing players to discontinue usage intention.

Keywords: Abandon, Mobile devices, Mobile social games, Perceived sacrifice.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 3577

Authors: Boyan Bontchev, Dessislava Vassileva

Abstract:

Internet computer games turn to be more and more attractive within the context of technology enhanced learning. Educational games as quizzes and quests have gained significant success in appealing and motivating learners to study in a different way and provoke steadily increasing interest in new methods of application. Board games are specific group of games where figures are manipulated in competitive play mode with race conditions on a surface according predefined rules. The article represents a new, formalized model of traditional quizzes, puzzles and quests shown as multimedia board games which facilitates the construction process of such games. Authors provide different examples of quizzes and their models in order to demonstrate the model is quite general and does support not only quizzes, mazes and quests but also any set of teaching activities. The execution process of such models is explained and, as well, how they can be useful for creation and delivery of adaptive e-learning courseware.

Keywords: Quiz, board game, e-learning, adaptive.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 2422

Authors: Anastasia Rita Widiarti, Phalita Nari Wastu

Abstract:

Hidden Markov Model (HMM) is a stochastic method which has been used in various signal processing and character recognition. This study proposes to use HMM to recognize Javanese characters from a number of different handwritings, whereby HMM is used to optimize the number of state and feature extraction. An 85.7 % accuracy is obtained as the best result in 16-stated vertical model using pure HMM. This initial result is satisfactory for prompting further research.

Keywords: Character recognition, off-line handwritingrecognition, Hidden Markov Model.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1988

Authors: Ehsan Amid, Sina Rezaei Aghdam

Abstract:

In this paper, a novel method for recognition of musical instruments in a polyphonic music is presented by using an embedded hidden Markov model (EHMM). EHMM is a doubly embedded HMM structure where each state of the external HMM is an independent HMM. The classification is accomplished for two different internal HMM structures where GMMs are used as likelihood estimators for the internal HMMs. The results are compared to those achieved by an artificial neural network with two hidden layers. Appropriate classification accuracies were achieved both for solo instrument performance and instrument combinations which demonstrates that the new approach outperforms the similar classification methods by means of the dynamic of the signal.

Keywords: hidden Markov model (HMM), embedded hidden Markov models (EHMM), MFCC, musical instrument.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1891

Authors: Boukelkoul Lahcen

Abstract:

The cumulative costs for O&M may represent as much as 65%-90% of the turbine's investment cost. Nowadays the cost effectiveness concept becomes a decision-making and technology evaluation metric. The cost of energy metric accounts for the effect replacement cost and unscheduled maintenance cost parameters. One key of the proposed approach is the idea of maintaining the WTs which can be captured via use of a finite state Markov chain. Such a model can be embedded within a probabilistic operation and maintenance simulation reflecting the action to be done. In this paper, an approach of estimating the cost of O&M is presented. The finite state Markov model is used for decision problems with number of determined periods (life cycle) to predict the cost according to various options of maintenance.

Keywords: Cost, finite state, Markov model, operation, maintenance.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1480

Authors: Jeng-Shin Sheu, Jenn-Kaie Lain, Tai-Kuo Woo, Jyh-Horng Wen

Abstract:

The least mean square (LMS) algorithmis one of the most well-known algorithms for mobile communication systems due to its implementation simplicity. However, the main limitation is its relatively slow convergence rate. In this paper, a booster using the concept of Markov chains is proposed to speed up the convergence rate of LMS algorithms. The nature of Markov chains makes it possible to exploit the past information in the updating process. Moreover, since the transition matrix has a smaller variance than that of the weight itself by the central limit theorem, the weight transition matrix converges faster than the weight itself. Accordingly, the proposed Markov-chain based booster thus has the ability to track variations in signal characteristics, and meanwhile, it can accelerate the rate of convergence for LMS algorithms. Simulation results show that the LMS algorithm can effectively increase the convergence rate and meantime further approach the Wiener solution, if the Markov-chain based booster is applied. The mean square error is also remarkably reduced, while the convergence rate is improved.

Keywords: LMS, Markov chain, convergence rate, accelerator.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1763

Authors: Nam Lethanh, Bryan T. Adey

Abstract:

In this paper, the potential use of an exponential hidden Markov model to model a hidden pavement deterioration process, i.e. one that is not directly measurable, is investigated. It is assumed that the evolution of the physical condition, which is the hidden process, and the evolution of the values of pavement distress indicators, can be adequately described using discrete condition states and modeled as a Markov processes. It is also assumed that condition data can be collected by visual inspections over time and represented continuously using an exponential distribution. The advantage of using such a model in decision making process is illustrated through an empirical study using real world data.

Keywords: Deterioration modeling, Exponential distribution, Hidden Markov model, Pavement management

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 2305

Authors: B. A. A. Paixão, J. V. B. Gomide

Abstract:

This work aims to analyze the locative structure used by the locative games of the company Niantic. To fulfill this objective, a literature review on the representation and simulation of cities was developed; interviews with Ingress players and playing Ingress. Relating these data, it was possible to deepen the relationship between the virtual and the real to create the simulation of cities and their cultural objects in locative games. Cities representation associates geo-location provided by the Global Positioning System (GPS), with augmented reality and digital image, and provides a new paradigm in the city interaction with its parts and real and virtual world elements, homeomorphic to real world. Bibliographic review of papers related to the representation and simulation study and their application in locative games was carried out and is presented in the present paper. The cities representation and simulation concepts in locative games, and how this setting enables the flow and immersion in urban space, are analyzed. Some examples of games are discussed for this new setting development, which is a mix of real and virtual world. Finally, it was proposed a Locative Structure for electronic games using the concepts of heterotrophic representations and isotropic representations conjoined with immediacy and hypermediacy.

Keywords: Cities representation, city simulation, games simulation, locative games.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 880

Authors: Kuo Hung Huang, Chong-Ji Ke

Abstract:

The purpose of this study is to explore the impacts of computer games on the mathematics instruction. First, the research designed and implemented the web-based games according to the content of existing textbook. And the researcher collected and analyzed the information related to the mathematics instruction integrating the computer games. In this study, the researcher focused on the learning motivation of mathematics, mathematics achievement, and pupil-teacher interactions in classroom. The results showed that students under instruction integrating computer games significantly improved in motivation and achievement. The teacher tended to use less direct teaching and provide more time for student-s active learning.

Keywords: computer games, mathematics instruction, pupil-teacher interaction, technology-enhanced learning

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1925

Authors: Fan-Yong Meng

Abstract:

In this paper, a generalized form of the Banzhaf-Owen value for cooperative fuzzy games with a coalition structure is proposed. Its axiomatic system is given by extending crisp case. In order to better understand the Banzhaf-Owen value for fuzzy games with a coalition structure, we briefly introduce the Banzhaf-Owen values for two special kinds of fuzzy games with a coalition structure, and give their explicit forms.

Keywords: Cooperative fuzzy game, Banzhaf-Owen value, multi linear extension, Choquet integral.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1553

Authors: Anita Kéri

Abstract:

Recent developments in the field of education have led to a renewed interest in teaching methodologies and practices. Gamification is fast becoming a key instrument in the education of new generations and besides other methods, serious games have become the center of attention. Ready-built serious games are available for most higher education institutions to buy and implement. However, monetary restraints and the unalterable nature of the games might deter most higher education institutions from the application of these serious games. Therefore, there is a continuously growing need for a customizable serious game that has been developed based on a concrete need analysis and experts’ opinion. There has been little evidence so far of serious games that have been created based on relevant and current need analysis from higher education institution teachers, professional practitioners and students themselves. Therefore, the aim of this current paper is to analyze the needs of higher education institution educators with special emphasis on their needs, the applicability of serious games in their classrooms, and exploring options for the development of a customizable serious game framework. The paper undertakes to analyze workshop discussions on implementing serious games in education and propose a customizable serious game framework applicable in the education of the new generation. Research results show that the most important feature of a serious game is its customizability. The fact that practitioners are able to manage different scenarios and upload their own content to a game seems to be a key to the increasingly widespread application of serious games in the classroom.

Keywords: Education, gamification, game-based learning, serious games.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 881

Authors: Vesna Bosilj Vuksic, Mirjana Pejic Bach, Tomislav Hernaus

Abstract:

The aim of this paper is to present a framework for empirical investigation of the effectiveness of simulation games for student learning of BPM concept. A future research methodology is explained and a normative model that extends the standard TAM model by introducing latent and mediating variables into the relationship between independent variables and dependent variable is developed. Future research propositions are defined in order to examine the benefits that can be achieved through the use of BPM simulation games in ERP courses.

Keywords: Business process management, simulation games, education, technology acceptance model.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 2570

Authors: Chun Wen, Tingzhu Huang

Abstract:

We present a new numerical method for the computation of the steady-state solution of Markov chains. Theoretical analyses show that the proposed method, with a contraction factor α, converges to the one-dimensional null space of singular linear systems of the form Ax = 0. Numerical experiments are used to illustrate the effectiveness of the proposed method, with applications to a class of interesting models in the domain of tandem queueing networks.

Keywords: Markov chains, tandem queueing networks, convergence, effectiveness.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1329

Authors: Terrence Chen, Thomas S. Huang

Abstract:

In this paper, we present the region based hidden Markov random field model (RBHMRF), which encodes the characteristics of different brain regions into a probabilistic framework for brain MR image segmentation. The recently proposed TV+L1 model is used for region extraction. By utilizing different spatial characteristics in different brain regions, the RMHMRF model performs beyond the current state-of-the-art method, the hidden Markov random field model (HMRF), which uses identical spatial information throughout the whole brain. Experiments on both real and synthetic 3D MR images show that the segmentation result of the proposed method has higher accuracy compared to existing algorithms.

Keywords: Finite Gaussian mixture model, Hidden Markov random field model, image segmentation, MRI.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 2101

Authors: Roslina Ibrahim, Azizah Jaafar

Abstract:

Educational games (EG) seem to have lots of potential due to digital games popularity and preferences of our younger generations of learners. However, most studies focus on game design and its effectiveness while little has been known about the factors that can affect users to accept or to reject EG for their learning. User acceptance research try to understand the determinants of information systems (IS) adoption among users by investigating both systems factors and users factors. Upon the lack of knowledge on acceptance factors for educational games, we seek to understand the issue. This study proposed a model of acceptance factors based on Unified Theory of Acceptance and Use of Technology (UTAUT). We use original model (performance expectancy, effort expectancy and social influence) together with two new determinants (learning opportunities and enjoyment). We will also investigate the effect of gender and gaming experience that moderate the proposed factors.

Keywords: educational games, games acceptance, user acceptance model, UTAUT

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 3708

Authors: Zöhre Kara Kayikci, Günther Palm

Abstract:

A word recognition architecture based on a network of neural associative memories and hidden Markov models has been developed. The input stream, composed of subword-units like wordinternal triphones consisting of diphones and triphones, is provided to the network of neural associative memories by hidden Markov models. The word recognition network derives words from this input stream. The architecture has the ability to handle ambiguities on subword-unit level and is also able to add new words to the vocabulary during performance. The architecture is implemented to perform the word recognition task in a language processing system for understanding simple command sentences like “bot show apple".

Keywords: Hebbian learning, hidden Markov models, neuralassociative memories, word recognition.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1523

Authors: Yi Yu, Lin Ma, Yong Sun, Yuantong Gu

Abstract:

This paper presents a novel method for remaining useful life prediction using the Elliptical Basis Function (EBF) network and a Markov chain. The EBF structure is trained by a modified Expectation-Maximization (EM) algorithm in order to take into account the missing covariate set. No explicit extrapolation is needed for internal covariates while a Markov chain is constructed to represent the evolution of external covariates in the study. The estimated external and the unknown internal covariates constitute an incomplete covariate set which are then used and analyzed by the EBF network to provide survival information of the asset. It is shown in the case study that the method slightly underestimates the remaining useful life of an asset which is a desirable result for early maintenance decision and resource planning.

Keywords: Elliptical Basis Function Network, Markov Chain, Missing Covariates, Remaining Useful Life

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1661

Authors: Tomáš Vencúrik, Jiří Nykodým

Abstract:

This study compares the intensity of game load among player positions and between the 1st and the 2nd half of the games. Two guards, three forwards, and three centers (female basketball players) participated in this study. The heart rate (HR) and its development were monitored during two competitive games. Statistically insignificant differences in the intensity of game load were recorded between guards, forwards, and centers below and above 85% of the maximal heart rate (HRmax) and in the mean HR as % of HRmax (87.81±3.79%, 87.02±4.37%, and 88.76±3.54%, respectively). Moreover, when the 1st and the 2nd half of the games were compared in the mean HR (87.89±4.18% vs. 88.14±3.63% of HRmax), no statistical significance was recorded. This information can be useful for coaching staff, to manage and to precisely plan the training process.

Keywords: Game load, heart rate, player positions, the 1st and the 2nd half of the games.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 2343

Authors: Alessandro Cincotti

Abstract:

In synchronized games players make their moves simultaneously rather than alternately. Synchronized Quadromineering is the synchronized version of Quadromineering, a variants of a classical two-player combinatorial game called Domineering. Experimental results for small m × n boards (with m + n < 15) and some theoretical results for general k × n boards (with k = 4, 5, 6) are presented. Moreover, some Synchronized Quadromineering variants are also investigated.

Keywords: Combinatorial games, Synchronized games, Quadromineering.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1205

Authors: Andrew Laghos

Abstract:

Computers are increasingly being used as educational tools in elementary/primary schools worldwide. A specific application of such computer use, is that of multimedia games, where the aim is to combine pedagogy and entertainment. This study reports on a case-study whereby an educational multimedia game has been developed for use by elementary school children. The stages of the application-s design, implementation and evaluation are presented. Strengths of the game are identified and discussed, and its weaknesses are identified, allowing for suggestions for future redesigns. The results show that the use of games can engage children in the learning process for longer periods of time with the added benefit of the entertainment factor.

Keywords: Education, entertainment, games, school

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 2245

Authors: N. Sopipan, A. Intarasit, K. Chuarkham

Abstract:

 In this paper, we forecast the volatility of Baht/USDs using Markov Regime Switching GARCH (MRS-GARCH) models. These models allow volatility to have different dynamics according to unobserved regime variables. The main purpose of this paper is to find out whether MRS-GARCH models are an improvement on the GARCH type models in terms of modeling and forecasting Baht/USD volatility. The MRS-GARCH is the best performance model for Baht/USD volatility in short term but the GARCH model is best perform for long term.

Keywords: Volatility, Markov Regime Switching, Forecasting.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1938

Authors: Alessandro Cincotti

Abstract:

The game of Maundy Block is the three-player variant of Maundy Cake, a classical combinatorial game. Even though to determine the solution of Maundy Cake is trivial, solving Maundy Block is challenging because of the identification of queer games, i.e., games where no player has a winning strategy.

Keywords: Combinatorial game, Maundy Cake, Three-player partizan games.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1516

Authors: Pavan K. Rallabandi, Kailash C. Patidar

Abstract:

In this paper, we present an optimization technique or a learning algorithm using the hybrid architecture by combining the most popular sequence recognition models such as Recurrent Neural Networks (RNNs) and Hidden Markov models (HMMs). In order to improve the sequence/pattern recognition/classification performance by applying a hybrid/neural symbolic approach, a gradient descent learning algorithm is developed using the Real Time Recurrent Learning of Recurrent Neural Network for processing the knowledge represented in trained Hidden Markov Models. The developed hybrid algorithm is implemented on automata theory as a sample test beds and the performance of the designed algorithm is demonstrated and evaluated on learning the deterministic finite state automata.

Keywords: Hybrid systems, Hidden Markov Models, Recurrent neural networks, Deterministic finite state automata.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 2884