Search results for: Questionnaire-based evaluation

Authors: Radoslav Delina, Michal Tkáč

Abstract:

Globalization, supported by information and communication technologies, changes the rules of competitiveness and increases the significance of information, knowledge and network cooperation. In line with this trend, the need for efficient trust-building tools has emerged. The absence of trust building mechanisms and strategies was identified within several studies. Through trust development, participation on e-business network and usage of network services will increase and provide to SMEs new economic benefits. This work is focused on effective trust building strategies development for electronic business network platforms. Based on trust building mechanism identification, the questionnairebased analysis of its significance and minimum level of requirements was conducted. In the paper, we are confirming the trust dependency on e-Skills which play crucial role in higher level of trust into the more sophisticated and complex trust building ICT solutions.

Keywords: Correlation analysis, decision trees, e-marketplace, trust building

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1927

Authors: Chao Qi, Fuyang Peng, Bo Deng, Xiaoyan Su

Abstract:

In the upgrade process of enterprise information systems, how to deal with and utilize those legacy systems affects the efficiency of construction and development of the new system. We propose an evaluation system, which comprehensively describes the capacity of legacy information systems in five aspects. Then we propose a practical legacy systems evaluation method. Base on the evaluation result, we can determine the current state of legacy system which was evaluated.

Keywords: Legacy Information Systems, Evaluation IndexSystem, Evaluation Method, Evaluation Level

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1620

Authors: Fatemeh Nikpay, Rodina Binti Ahmad, Babak Darvish Rouhani

Abstract:

Enterprise Architecture (EA) is employed by enterprises for providing integrated Information Systems (ISs) in order to support alignment of their business and Information Technology (IT). Evaluation of EA implementation can support enterprise to reach intended goals. There are some problems in current evaluation methods of EA implementation that lead to ineffectiveness implementation of EA. This paper represents current issues on evaluation of EA implementation. In this regard, we set the framework in order to represent evaluation’s issues based on their functionality and structure. The results of this research not only increase the knowledge of evaluation, but also could be useful for both academics and practitioners in order to realize the current situation of evaluations.

Keywords: Current issues on EA, implementation evaluation, Evaluation, Enterprise Architecture, Evaluation of Enterprise Architecture Implementation.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 4011

Authors: Tomas Luneckas, Dainius Udris

Abstract:

In this paper a simple terrain evaluation method for hexapod robot is introduced. This method is based on feet coordinate evaluation when all are on the ground. Depending on the feet coordinate differences the local terrain evaluation is possible. Terrain evaluation is necessary for right gait selection and/or body position correction. For terrain roughness evaluation three planes are plotted: two of them as definition points use opposite feet coordinates, third coincides with the robot body plane. The leaning angle of body plane is evaluated measuring gravity force using three-axis accelerometer. Terrain roughness evaluation method is based on angle estimation between normal vectors of these planes. Aim of this work is to present a simple method for embedded robot controller, allowing to find the best further movement settings.

Keywords: Hexapod robot, pose estimation, terrain evaluation, terrain roughness.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1802

Authors: T. Lantharthong, N. Phanthuna

Abstract:

This paper presents reliability evaluation techniques which are applied in distribution system planning studies and operation. Reliability of distribution systems is an important issue in power engineering for both utilities and customers. Reliability is a key issue in the design and operation of electric power distribution systems and load. Reliability evaluation of distribution systems has been the subject of many recent papers and the modeling and evaluation techniques have improved considerably.

Keywords: Reliability Evaluation, Optimization Technique, Reliability Indices

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 4566

Authors: Jeongim Kang, Moon Hee Kim, Seong Baeg Kim

Abstract:

This paper proposes a novel e-learning model that is  based on a dynamic formative evaluation. On evaluating the existing  format of e-learning, conditions regarding repetitive learning to  achieve mastery, causes issues for learners to lose tension and become  neglectful of learning. The dynamic formative evaluation proposed is  able to supplement limitation of the existing approaches. Since a  repetitive learning method does not provide a perfect feedback, this  paper puts an emphasis on the dynamic formative evaluation that is  able to maximize learning achievement. Through the dynamic  formative evaluation, the instructor is able to refer to the evaluation  result when making an estimation about the learner. To show the flow  chart of learning, based on the dynamic formative evaluation, the  model proves its effectiveness and validity.

 

Keywords: Online learning, dynamic formative evaluation, mastery learning, repetitive learning method, learning achievement.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1782

Authors: Ivayla Trifonova, Naoum Jamous, Holger Schrödl

Abstract:

A good web design is a prerequisite for a successful business nowadays, especially since the internet is the most common way for people to inform themselves. Web design includes the optical composition, the structure, and the user guidance of websites. The importance of each website leads to the question if there is a way to measure its usefulness. The aim of this paper is to suggest a methodology for the evaluation of web design. The desired outcome is to have an evaluation that is concentrated on a specific website and its target group.

Keywords: Evaluation methodology, factor analysis, target group, web design.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1266

Authors: Nicoletta Adamo-Villani

Abstract:

The object of this research is the design and evaluation of an immersive Virtual Learning Environment (VLE) for deaf children. Recently we have developed a prototype immersive VR game to teach sign language mathematics to deaf students age K- 4 [1] [2]. In this paper we describe a significant extension of the prototype application. The extension includes: (1) user-centered design and implementation of two additional interactive environments (a clock store and a bakery), and (2) user-centered evaluation including development of user tasks, expert panel-based evaluation, and formative evaluation. This paper is one of the few to focus on the importance of user-centered, iterative design in VR application development, and to describe a structured evaluation method.

Keywords: 3D Animation, Virtual Reality, Virtual Learning Environments, User-Centered Design, User-centered Evaluation.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 2203

Authors: Shigeaki Sakurai, Makino Kyoko, Shigeru Matsumoto

Abstract:

This paper proposes a method that predicts attractive evaluation objects. In the learning phase, the method inductively acquires trend rules from complex sequential data. The data is composed of two types of data. One is numerical sequential data. Each evaluation object has respective numerical sequential data. The other is text sequential data. Each evaluation object is described in texts. The trend rules represent changes of numerical values related to evaluation objects. In the prediction phase, the method applies new text sequential data to the trend rules and evaluates which evaluation objects are attractive. This paper verifies the effect of the proposed method by using stock price sequences and news headline sequences. In these sequences, each stock brand corresponds to an evaluation object. This paper discusses validity of predicted attractive evaluation objects, the process time of each phase, and the possibility of application tasks.

Keywords: Trend rule, frequent pattern, numerical sequential data, text sequential data, evaluation object.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1235

Authors: Candy P. Schulze, Michael Brieke, Prof. Peter Nyhuis

Abstract:

The building of a factory can be a strategic investment owing to its long service life. An evaluation that only focuses, for example, on payments for the building, the technical equipment of the factory, and the personnel for the enterprise is – considering the complexity of the system factory – not sufficient for this long-term view. The success of an investment is secured, among other things, by the attainment of nonmonetary goals, too, like transformability. Such aspects are not considered in traditional investment calculations like the net present value method. This paper closes this gap with the enhanced economic evaluation (EWR) for factory planning. The procedure and the first results of an application in a project are presented.

Keywords: economic efficiency, holistic evaluation, factory planning

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1696

Authors: Alexandru Balog, Costin Pribeanu, Dragos Iordache

Abstract:

Formative usability evaluation aims at finding usability problems during the development process. The earlier these problems are identified, the less expensive to fix they are. This paper presents some preliminary results from a formative usability testing of the 1st prototype developed for the ARiSE (Augmented Reality in School Environments) project.

Keywords: AR-based educational systems, formative evaluation, usability evaluation, user testing.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 2551

Authors: Guangzhuang Yu, Chun-Chu Liu

Abstract:

With the advent of the Internet era in recent years, social music education has gradually shifted from the original entity education mode to the mode of entity plus network teaching. No matter for school music education, professional music education or social music education, the teaching quality is the most important evaluation index. Regarding the research on teaching quality evaluation, scholars at home and abroad have contributed a lot of research results on the basis of multiple methods and evaluation subjects. However, to our best knowledge the complete evaluation model for the virtual teaching interaction mode of the emerging network music education Application (APP) has not been established. This research firstly found out the basic dimensions that accord with the teaching quality required by the three parties, constructing the quality evaluation index system; and then, on the basis of expounding the connotation of each index, it determined the weight of each index by using method of fuzzy analytic hierarchy process, providing ideas and methods for scientific, objective and comprehensive evaluation of the teaching quality of network education APP.

Keywords: Network music education APP, teaching quality evaluation, index, connotation.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 845

Authors: J. Beel, B. Gipp

Abstract:

Research papers are usually evaluated via peer review. However, peer review has limitations in evaluating research papers. In this paper, Scienstein and the new idea of 'collaborative document evaluation' are presented. Scienstein is a project to evaluate scientific papers collaboratively based on ratings, links, annotations and classifications by the scientific community using the internet. In this paper, critical success factors of collaborative document evaluation are analyzed. That is the scientists- motivation to participate as reviewers, the reviewers- competence and the reviewers- trustworthiness. It is shown that if these factors are ensured, collaborative document evaluation may prove to be a more objective, faster and less resource intensive approach to scientific document evaluation in comparison to the classical peer review process. It is shown that additional advantages exist as collaborative document evaluation supports interdisciplinary work, allows continuous post-publishing quality assessments and enables the implementation of academic recommendation engines. In the long term, it seems possible that collaborative document evaluation will successively substitute peer review and decrease the need for journals.

Keywords: Peer Review, Alternative, Collaboration, Document Evaluation, Rating, Annotations.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1489

Authors: Juraj Camaj, Martin Kendra, Jaroslav Masek

Abstract:

The paper is focused to the evaluation railway tracks in the Slovakia by using Multi-Criteria method. Evaluation of railway tracks has important impacts for the assessment of investment in technical equipment. Evaluation of railway tracks also has an important impact for the allocation of marshalling yards. Marshalling yards are in transport model as centers for the operation assigned catchment area. This model is one of the effective ways to meet the development strategy of the European Community's railways. By applying this model in practice, a transport company can guarantee a higher quality of service and then expect an increase in performance. The model is also applicable to other rail networks. This model supplements a theoretical problem of train formation problem of new ways of looking at evaluation of factors affecting the organization of wagon flows.

Keywords: Railway track, multi-criteria methods, evaluation, transportation model.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 2212

Authors: Yao Jian-Min, Lv Qiang, Zhang Jing

Abstract:

Many measures have been proposed for machine translation evaluation (MTE) while little research has been done on the performance of MTE methods. This paper is an effort for MTE performance analysis. A general frame is proposed for the description of the MTE measure and the test suite, including whether the automatic measure is consistent with human evaluation, whether different results from various measures or test suites are consistent, whether the content of the test suite is suitable for performance evaluation, the degree of difficulty of the test suite and its influence on the MTE, the relationship of MTE result significance and the size of the test suite, etc. For a better clarification of the frame, several experiment results are analyzed relating human evaluation, BLEU evaluation, and typological MTE. A visualization method is introduced for better presentation of the results. The study aims for aid in construction of test suite and method selection in MTE practice.

Keywords: Machine translation, natural language processing, visualization.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1706

Authors: Chao Qi, Fuyang Peng, Bo Deng, Xiaoyan Su

Abstract:

In the upgrade process of enterprise information systems, whether new systems will be success and their development will be efficient, depends on how to deal with and utilize those legacy systems. We propose an evaluation system, which comprehensively describes the capacity of legacy information systems in five aspects. Then a practical legacy systems evaluation method is scripted. Base on the evaluation result, we put forward 4 kinds of migration strategy: eliminated, maintenance, modification, encapsulating. The methods and strategies play important roles in practice.

Keywords: Legacy Systems, Evaluation Method, Migration Strategy.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1712

Authors: Hao Hsi Tseng

Abstract:

Taiwan’s infrastructure is gradually deteriorating, while resources for maintenance and replacement are increasingly limited, raising the urgent need for methods for maintaining existing infrastructure within constrained budgets. Infrastructure value evaluation is used to enhance the efficiency of infrastructure maintenance work, allowing administrators to quickly assess the maintenance needs and performance by observing variation in infrastructure value. This research establishes a value evaluation model for Taiwan’s highway box girder bridges. The operating mechanism and process of the model are illustrated in a practical case.

Keywords: Box girder bridge, deterioration, infrastructure, maintenance, value evaluation.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1236

Authors: Badiossadat Hassanpour, Nangkula Utaberta, Azami Zaharim, Nurakmal Goh Abdullah

Abstract:

Architecture education was based on apprenticeship models and its nature has not changed much during long period but the Source of changes was its evaluation process and system. It is undeniable that art and architecture education is completely based on transmitting knowledge from instructor to students. In contrast to other majors this transmitting is by iteration and practice and studio masters try to control the design process and improving skills in the form of supervision and criticizing. Also the evaluation will end by giving marks to students- achievements. Therefore the importance of the evaluation and assessment role is obvious and it is not irrelevant to say that if we want to know about the architecture education system, we must first study its assessment procedures. The evolution of these changes in western countries has literate and documented well. However it seems that this procedure has unregarded in Malaysia and there is a severe lack of research and documentation in this area. Malaysia as an under developing and multicultural country which is involved different races and cultures is a proper origin for scrutinizing and understanding the evaluation systems and acceptability amount of current implemented models to keep the evaluation and assessment procedure abreast with needs of different generations, cultures and even genders. This paper attempts to answer the questions of how evaluation and assessments are performed and how students perceive this evaluation system in the context Malaysia. The main advantage of this work is that it contributes in international debate on evaluation model.

Keywords: Architecture, assessment, design studio, learning

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 2878

Authors: Nan Li, Ying Wang

Abstract:

Using entropy weight and TOPSIS method, a comprehensive evaluation is done on the development level of Chinese regional service industry in this paper. Firstly, based on existing research results, an evaluation index system is constructed from the scale of development, the industrial structure and the economic benefits. An evaluation model is then built up based on entropy weight and TOPSIS, and an empirical analysis is conducted on the development level of service industries in 31 Chinese provinces during 2006 and 2009 from the two dimensions or time series and cross section, which provides new idea for assessing regional service industry. Furthermore, the 31 provinces are classified into four categories based on the evaluation results, and deep analysis is carried out on the evaluation results.

Keywords: Chinese regional service industry, Development level, Entropy weight, TOPSIS Evaluation Method

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1506

Authors: S. Samanta, D. Datta, S. S. Gautam

Abstract:

In this paper, quantitative evaluation of ultrasonic Cscan images through estimation of their Fractal Dimension (FD) is discussed. Necessary algorithm for evaluation of FD of any 2-D digitized image is implemented by developing a computer code. For the evaluation purpose several C-scan images of the Kevlar composite impacted by high speed bullet and glass fibre composite having flaw in the form of inclusion is used. This analysis automatically differentiates a C-scan image showing distinct damage zone, from an image that contains no such damage.

Keywords: C-scan, Impact, Fractal Dimension, Kevlar composite and Inclusion Flaw

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1744

Authors: Dilek Özdemir, Gül Tekin Temur

Abstract:

In Supply Chain Management (SCM), strengthening partnerships with suppliers is a significant factor for enhancing competitiveness. Hence, firms increasingly emphasize supplier evaluation processes. Supplier evaluation systems are basically developed in terms of criteria such as quality, cost, delivery, and flexibility. Because there are many variables to be analyzed, this process becomes hard to execute and needs expertise. On this account, this study aims to develop an expert system on supplier evaluation process by designing Artificial Neural Network (ANN) that is supported with Data Envelopment Analysis (DEA). The methods are applied on the data of 24 suppliers, which have longterm relationships with a medium sized company from German Iron and Steel Industry. The data of suppliers consists of variables such as material quality (MQ), discount of amount (DOA), discount of cash (DOC), payment term (PT), delivery time (DT) and annual revenue (AR). Meanwhile, the efficiency that is generated by using DEA is added to the supplier evaluation system in order to use them as system outputs.

Keywords: Artificial Neural Network (ANN), DataEnvelopment Analysis (DEA), Supplier Evaluation System.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 2153

Authors: C. Ardil

Abstract:

In this research, the Balkan peninsula countries' developmental integration into European Union represents the strategic economic development objectives of the countries in the region. In order to objectively analyze the level of economic development competition of Balkan Peninsula countries, the mathematical compromise programming technique of multicriteria evaluation is used in this ranking problem. The primary aim of this research is to explain the role and significance of the multicriteria method evaluation using a real example of compromise solutions. Using the mathematical compromise programming technique, twelve countries of the Balkan Peninsula are economically evaluated and mutually compared. The economic development evaluation of the countries is performed according to five evaluation criteria forming the basis for economic development evaluation. The multiattribute model is solved using the mathematical compromise programming technique for producing different Pareto solutions. The results obtained by the multicriteria evaluation gives the possibility of identification and evaluation of the most eminent economic development indicators for each country separately. Finally, in this way, the proposed method has proved to be a successful model for the evaluation of the Balkan peninsula countries' economic development competition.

Keywords: Balkan peninsula countries, standard deviation, multicriteria decision making, mathematical compromise programming, multicriteria decision making, multicriteria analysis, multicriteria decision analysis.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 794

Authors: Mihai Caramihai, Irina Severin

Abstract:

Despite the extensive use of eLearning systems, there is no consensus on a standard framework for evaluating this kind of quality system. Hence, there is only a minimum set of tools that can supervise this judgment and gives information about the course content value. This paper presents two kinds of quality set evaluation indicators for eLearning courses based on the computational process of three known metrics, the Euclidian, Hamming and Levenshtein distances. The “distance" calculus is applied to standard evaluation templates (i.e. the European Commission Programme procedures vs. the AFNOR Z 76-001 Standard), determining a reference point in the evaluation of the e-learning course quality vs. the optimal concept(s). The case study, based on the results of project(s) developed in the framework of the European Programme “Leonardo da Vinci", with Romanian contractors, try to put into evidence the benefits of such a method.

Keywords: eLearning, European programme, metrics, quality evaluation

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1519

Authors: Kamalendu Pal

Abstract:

This paper describes the main features of a knowledge-based system evaluation method. System evaluation is placed in the context of a hybrid legal decision-support system, Advisory Support for Home Settlement in Divorce (ASHSD). Legal knowledge for ASHSD is represented in two forms, as rules and previously decided cases. Besides distinguishing the two different forms of knowledge representation, the paper outlines the actual use of these forms in a computational framework that is designed to generate a plausible solution for a given case, by using rule-based reasoning (RBR) and case-based reasoning (CBR) in an integrated environment. The nature of suitability assessment of a solution has been considered as a multiple criteria decision-making process in ASHAD evaluation. The evaluation was performed by a combination of discussions and questionnaires with different user groups. The answers to questionnaires used in this evaluations method have been measured as a fuzzy linguistic term. The finding suggests that fuzzy linguistic evaluation is practical and meaningful in knowledge-based system development purpose. 

Keywords: Case-based reasoning, decision-support system, fuzzy linguistic term, rule-based reasoning, system evaluation.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1632

Authors: A.R Alesaadi, M. Nafar, A.H. Gheisari

Abstract:

In this paper sensitivity analysis is performed for reliability evaluation of power systems. When examining the reliability of a system, it is useful to recognize how results change as component parameters are varied. This knowledge helps engineers to understand the impact of poor data, and gives insight on how reliability can be improved. For these reasons, a sensitivity analysis can be performed. Finally, a real network was used for testing the presented method.

Keywords: sensitivity analysis, reliability evaluation, powersystems.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 2273

Authors: Emre Alptekin, Damla Yalçınyiğit, Gülfem Alptekin

Abstract:

In highly competitive environments, a growing number of companies must regularly launch new products speedily and successfully. A company-s success is based on the systematic, conscious product designing method which meets the market requirements and takes risks as well as resources into consideration. Research has found that developing and launching new products are inherently risky endeavors. Hence in this research, we aim at introducing a risk evaluation framework for the new product innovation process. Our framework is based on the fuzzy analytical hierarchy process (FAHP) methodology. We have applied all the stages of the framework on the risk evaluation process of a pharmaceuticals company.

Keywords: Evaluation, risks, product innovation.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1492

Authors: M.A. Lazim, M. Suriani

Abstract:

Knowing consumers' preferences and perceptions of the sensory evaluation of drink products are very significant to manufacturers and retailers alike. With no appropriate sensory analysis, there is a high risk of market disappointment. This paper aims to rank the selected coffee products and also to determine the best of quality attribute through sensory evaluation using fuzzy decision making model. Three products of coffee drinks were used for sensory evaluation. Data were collected from thirty judges at a hypermarket in Kuala Terengganu, Malaysia. The judges were asked to specify their sensory evaluation in linguistic terms of the quality attributes of colour, smell, taste and mouth feel for each product and also the weight of each quality attribute. Five fuzzy linguistic terms represent the quality attributes were introduced prior analysing. The judgment membership function and the weights were compared to rank the products and also to determine the best quality attribute. The product of Indoc was judged as the first in ranking and 'taste' as the best quality attribute. These implicate the importance of sensory evaluation in identifying consumers- preferences and also the competency of fuzzy approach in decision making.

Keywords: fuzzy decision making, fuzzy linguistic, membership function, sensory evaluation,

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 2780

Authors: Heungsu Lee, Youngseok Kim, Jonghwa Yi, Chul Park

Abstract:

A landing pier is subjected to safety assessment by visual inspection and design data, but it is difficult to check the damage in real-time. In this study, real - time damage detection and safety evaluation methods were studied. As a result of structural analysis of the arbitrary landing pier structure, the inflection point of deformation and moment occurred at 10%, 50%, and 90% of pile length. The critical value of Fiber Bragg Grating (FBG) sensor was set according to the safety factor, and the FBG sensor application method for real - time safety evaluation was derived.

Keywords: FBG sensor, harbor structure, maintenance, safety evaluation system.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 936

Authors: Ong Sing Goh, C. Ardil, Wilson Wong, Chun Che Fung

Abstract:

The evaluation of conversational agents or chatterbots question answering systems is a major research area that needs much attention. Before the rise of domain-oriented conversational agents based on natural language understanding and reasoning, evaluation is never a problem as information retrieval-based metrics are readily available for use. However, when chatterbots began to become more domain specific, evaluation becomes a real issue. This is especially true when understanding and reasoning is required to cater for a wider variety of questions and at the same time to achieve high quality responses. This paper discusses the inappropriateness of the existing measures for response quality evaluation and the call for new standard measures and related considerations are brought forward. As a short-term solution for evaluating response quality of conversational agents, and to demonstrate the challenges in evaluating systems of different nature, this research proposes a blackbox approach using observation, classification scheme and a scoring mechanism to assess and rank three example systems, AnswerBus, START and AINI.

Keywords: Evaluation, conversational agents, Response Quality, chatterbots

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1927

Authors: Luis Teran, Aleksandar Drobnjak

Abstract:

The number of electronic participation (eParticipation) projects introduced by different governments and international organisations is considerably high and increasing. In order to have an overview of the development of these projects, various evaluation frameworks have been proposed. In this paper, a five-level participation model, which takes into account the advantages of the Social Web or Web 2.0, together with a quantitative approach for the evaluation of eParticipation projects is presented. Each participation level is evaluated independently, taking into account three main components: Web evolution, media richness, and communication channels. This paper presents the evaluation of a number of existing Voting Advice Applications (VAAs). The results provide an overview of the main features implemented by each project, their strengths and weaknesses, and the participation levels reached.

Keywords: Evaluation Framework, eParticipation, e-Participation, Electronic Participation, eDemocracy, e-Democracy, Electronic Democracy, Voting Advice Applications, VAAs.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 2656