Search results for: rasch%20analysis

Authors: Paramita Sekar Ayu, Sunjaya Deni Kurniadi, Yamazaki Chiho, Hilfi Lukman, Koyama Hiroshi

Abstract:

Social cohesion, or closeness among members of society, is an important determinant of population health. A cohesive society is a crucial societal condition for a positive life evaluation and subjective wellbeing, and people living in a cohesive society are happier and more satisfied with life and achieve better health status. The objective of this study was to compose and validate a questionnaire for measuring social cohesion with Rasch analysis. We develop a set of 13 questions to measure 4 dimensions of social cohesion. Random samples of 166 Bandung citizens’ were selected to answer the questionnaire. To evaluate the questionnaire’s validity and reliability, Rasch analysis (a psychometric model for analyzing categorical data on questionnaire responses) was carried out using Winsteps version 3.75.0. Rasch analysis was performed on the response given to 13 items included in the questionnaire. The reliability coefficient, Cronbach’s alpha was 0.70, model RMSE 0.08, SD 0.54, separation 7.14, and reliability of 0.98. ‘Kohesif-Ques’ is a useful instrument to assess social cohesion.

Keywords: rasch analysis, rasch model, social cohesion, quesionnaire

Procedia PDF Downloads 130

Authors: Austyn Snowden

Abstract:

Aim: To examine the construct validity of the Trait Emotional Intelligence Questionnaire Short form. Background: Emotional intelligence involves the identification and regulation of our own emotions and the emotions of others. It is therefore a potentially useful construct in the investigation of recruitment and retention in nursing and many questionnaires have been constructed to measure it. Design: Secondary analysis of existing dataset of responses to TEIQue-SF using concurrent application of Rasch analysis and confirmatory factor analysis. Method: First year undergraduate nursing and computing students completed Trait Emotional Intelligence Questionnaire-Short Form. Responses were analysed by synthesising results of Rasch analysis and confirmatory factor analysis.

Keywords: emotional intelligence, rasch analysis, factor analysis, nurse recruiters

Procedia PDF Downloads 429

Authors: Rizki Nor Amelia, Farida A. Setiawati

Abstract:

This study aimed to: (1) describe the characteristics of teacher-made test instrument used to measure the ability of students’chemistry, and (2) identify the presence of the compability difficulty level set by teachers to difficulty level by empirical results. Based on these objectives, this study was a descriptive research. The analysis in this study used the Rasch model and Chi-square statistics. Analysis using Rasch Model was based on the response patterns of high school students to the teacher-made test instrument on chemistry subject Academic Year 2015/2016 in the Yogyakarta. The sample of this research were 358 students taken by cluster random sampling technique. The analysis showed that: (1) a teacher-made tests instrument has a medium on the mean difficulty level. This instrument is capable to measure the ability on the interval of -0,259 ≤ θ ≤ 0,659 logit. Maximum Test Information Function obtained at 18.187 on the ability +0,2 logit; (2) 100% items categorized either as easy or difficult by rasch model is match with the teachers’ judgment; while 37 items are categorized according to rasch model which 8.10% and 10.81% categorized as easy and difficult items respectively according to the teachers, the others are medium categorized. Overall, the distribution of the level of difficulty formulated by the teachers has the distinction (not match) to the level of difficulty based on the empirical results.

Keywords: chemistry, items parameter fitness, Rasch model, teacher-made test

Procedia PDF Downloads 195

Authors: Wan Nur'ashiqin Wan Mohamad, Zarina Othman, Mohd Azman Abas, Azizah Ya'acob, Rozmel Abdul Latiff

Abstract:

Effective doctor-patient interaction is vital to both doctor and patient relationship. It is the cornerstone of good practice and an integral quality of a healthcare institution. This paper presented the hierarchy of the communication elements in doctor-patient interaction during medical consultations in a medical centre in Malaysia. This study adapted The Picker Patient Experience Questionnaire (2002) to obtain the information from patients. The questionnaire survey was responded by 100 patients between the ages of 20 and 50. Data collected were analysed using Rasch Measurement Model to yield the hierarchy of the communication elements in doctor-patient interaction. The findings showed that the three highest ranking on the doctor-patient interaction were doctor’s treatment, important information delivery and patient satisfaction of doctor’s responses. The results are valuable in developing the framework for communication ethics of doctors.

Keywords: communication elements, doctor-patient interaction, hierarchy, Rasch measurement model

Procedia PDF Downloads 135

Authors: Nurfirdawati Muhamad Hanafi, Azmanirah Ab Rahman, Marina Ibrahim Mukhtar, Jamil Ahmad, Sarebah Warman

Abstract:

This study was conducted to generate empirical evidence on validity and reliability of the item of Competency Assessment Implementation (CAI) Instrument using Rasch Model for polythomous data aided by Winstep software version 3.68. The construct validity was examined by analyzing the point-measure correlation index (PTMEA), in fit and outfit MNSQ values; meanwhile the reliability was examined by analyzing item reliability index. A survey technique was used as the major method with the CAI instrument on 156 teachers from vocational schools. The results have shown that the reliability of CAI Instrument items were between 0.80 and 0.98. PTMEA Correlation is in positive values, in which the item is able to distinguish between the ability of the respondent. Statistical data obtained shows that out of 154 items, 12 items from the instrument suggested to be omitted. This study is hoped could bring a new direction to the process of data analysis in educational research.

Keywords: competency assessment, reliability, validity, item analysis

Procedia PDF Downloads 401

Authors: L. D. Upegui, Isabel Alvarez-Solorza, Karina Garduno-Ulloa, Maren Boecker

Abstract:

Introduction: The increasing prevalence rate of resistant and multiresistant bacterial strains to antibiotics is a threat to public health and requires a rapid multifunctional answer. Individuals that are affected by resistant strains present a higher morbidity and mortality than individuals that are infected with the same species of bacteria but with sensitive strains. There have been identified risk factors that are related to the misuse and overuse of antibiotics, like socio-demographic characteristics and psychological aspects of the individuals that have not been explored objectively due to a lack of valid and reliable instruments for their measurement. Objective: To validate a questionnaire for the evaluation of the levels of knowledge related to the use of antibiotics in a Mexican population. Materials and Methods: Analytical cross-sectional observational study. The questionnaire consists of 12 items to evaluated knowledge (1=no, 2=not sure, 3=yes) regarding the use of antibiotics, with higher scores corresponding to a higher level of knowledge. Data are collected in a sample of students. Data collection is still ongoing. In this abstract preliminary results of 30 respondents are reported which were collected during pilot-testing. The validation of the instrument was done using the Rasch model. Fit to the Rasch model was tested checking overall fit to the model, unidimensionality, local independence and evaluating the presence of Differential Item Functioning (DIF) by age and gender. The software Rumm2030 and the SPSS were used for the analyses. Results: The participants of the pilot-testing presented an average age of 32 years ± 12.6 and 53% were women. The preliminary results indicated that the items showed good fit to the Rasch model (chi-squared=12.8 p=0.3795). Unidimensionality (number of significant t-tests of 3%) could be proven, the items were locally independent, and no DIF was observed. Knowledge was the smallest regarding statements on the role of antibiotics in treating infections, e.g., most of the respondents did not know that antibiotics would not work against viral infections (70%) and that they could also cause side effects (87%). The knowledge score ranged from 0 to 100 points with a transformed measurement (mean of knowledge 27.1 ± 4.8). Conclusions: The instrument showed good psychometric proprieties. The low scores of knowledge about antibiotics suggest that misinterpretations on the use of these medicaments were prevalent, which could influence the production of antibiotic resistance. The application of this questionnaire will allow the objective identification of 'Hight risk groups', which will be the target population for future educational campaigns, to reduce the knowledge gaps on the general population as an effort against antibiotic resistance.

Keywords: antibiotics, knowledge, misuse, overuse, questionnaire, Rasch model, validation

Procedia PDF Downloads 116

Authors: Chia-Chun Wu, Ying-Yao Cheng

Abstract:

The purpose of this study was developing the 'short-form BASIC scale' psychotic tendencies subscale so as to provide a more efficient, economical and effective way to assess the mental health of recruits. 1749 students from Naval Recruit Training Center participated in this study. The multidimensional constructs of psychotic tendencies subscale include four dimensions: schizophrenic tendencies, manic tendencies, depression tendencies, and suicidal ideation. We cut down the 36-item psychotic tendencies subscale to 25 items by using multidimension Rasch techniques. They were applied to assess model-data fit and to provide the validity evidence of the short form BASIC scale of psychotic tendencies subscale. The person separation reliabilities of the measures from four dimensions were .70, .67, .74 and .57, respectively. In addition, there is a notable correlation between the length version and short version of schizophrenic tendencies (scaled .89), manic tendencies (.96), depression tendencies (.97) and suicidal ideation (.97). The results have indicated that the development of the study of short-form scale sufficient to replace the original scale. Therefore, it is suggested that short-form basic scale is used to assess the mental health with participants being more willing to answer questions to ensure the validation of assessments.

Keywords: BASIC scale, military, Rasch analysis, short-form scale

Procedia PDF Downloads 328

Authors: Xiaoting Shi, Xiaomei Ma

Abstract:

In higher education, students' self-assessment and peer assessment of compositions in writing classes can effectively improve their ability of evaluative judgment. However, students' self-assessment and peer assessment are not advocated by most teachers because of the significant difference in scoring compared with teacher assessment. This study used a multi-faceted Rasch model to explore whether an English writing checklist containing 30 descriptors can effectively improve rating consistency among self-assessment, peer assessment and teacher assessment. Meanwhile, a questionnaire was adopted to survey students’ and teachers’ attitudes toward self-assessment and peer assessment using the writing checklist. Results of the multi-faceted Rasch model analysis show that the writing checklist can effectively distinguish the students’ writing ability (separate coefficient = 2.05, separate reliability = 0.81, chi-square value (df = 32) = 123.4). Moreover, the results revealed that the checklist could improve rating consistency among self-assessment, peer assessment and teacher assessment. (separate coefficient = 1.71, separate reliability = 0.75, chi-square value (df=4) = 20.8). The results of the questionnaire showed that more than 85% of students and all teachers believed that the checklist had a good advantage in self-assessment and peer assessment, and they were willing to use the checklist to conduct self-assessment and peer assessment in class in the future.

Keywords: english writing, self-assessment, peer assessment, writing checklist

Procedia PDF Downloads 121

Authors: Yeşim Özer Özkan, Fatma Büşra Fincan

Abstract:

Biased sample is a regression of an observation, design process and all of the specifications lead to tendency of a side or the situation of leaving from the objectivity. It is expected that, test items are answered by the students who come from different social groups and the same ability not to be different from each other. The importance of the expectation increases especially during student selection and placement examinations. For example, all of the test items should not be beneficial for just a male or female group. The aim of the research is an investigation of item bias whether or not the exam included in 2014 free boarding and scholarship examination in terms of gender variable. Data which belong to 5th, 6th, and 7th grade the secondary education students were obtained by the General Directorate of Measurement, Evaluation and Examination Services in Turkey. 20% students were selected randomly within 192090 students. Based on 38418 students’ exam paper were examined for determination item bias. Winsteps 3.8.1 package program was used to determine bias in analysis of data, according to Rasch Model in respect to gender variable. Mathematics items tests were examined in terms of gender bias. Firstly, confirmatory factor analysis was applied twenty-five math questions. After that, NFI, TLI, CFI, IFI, RFI, GFI, RMSEA, and SRMR were examined in order to be validity and values of goodness of fit. Modification index values of confirmatory factor analysis were examined and then some of the items were omitted because these items gave an error in terms of model conformity and conceptual. The analysis shows that in 2014 free boarding and scholarship examination exam does not include bias. This is an indication of the gender of the examination to be made in favor of or against different groups of students.

Keywords: gender, item bias, placement test, Rasch model

Procedia PDF Downloads 201

Authors: Zülfiye Bıkmaz, Aytolan Yıldırım

Abstract:

This study was planned as a descriptive regressive study in order to determine the relationship between the competency levels of working nurses, the levels of competency expected by nursing students, the critical thinking disposition of nurses, their perceived autonomy levels, and certain socio demographic characteristics. It is also a methodological study with regard to the intercultural adaptation of the Nursing Competence Scale (NCS) in both working and student samples. The sample of the study group of nurses at a university hospital for at least 6 months working properly and consists of 443 people filled out questionnaires. The student group, consisting of 543 individuals from the 4 public university nursing 3rd and 4th grade students. Data collection tools consisted of a questionnaire prepared in order to define the socio demographic, economic, and personal characteristics of the participants, the ‘Nursing Competency Scale’, the ‘Autonomy Subscale of the Sociotropy – Autonomy Scale’, and the ‘California Critical Thinking Disposition Inventory’. In data evaluation, descriptive statistics, nonparametric tests, Rasch analysis and correlation and regression tests were used. The language validity of the ‘NCS’ was performed by translation and back translation, and the context validity of the scale was performed with expert views. The scale, which was formed into its final structure, was applied in a pilot application from a group consisting of graduate and student nurses. The time constancy of the test was obtained by analysis testing retesting method. In order to reduce the time problems with the two half reliability method was used. The Cronbach Alfa coefficient of the scale was found to be 0.980 for the nurse group and 0.986 for the student group. Statistically meaningful relationships between competence and critical thinking and variables such as age, gender, marital status, family structure, having had critical thinking training, education level, class of the students, service worked in, employment style and position, and employment duration were found. Statistically meaningful relationships between autonomy and certain variables of the student group such as year, employment status, decision making style regarding self, total duration of employment, employment style, and education status were found. As a result, it was determined that the NCS which was adapted interculturally was a valid and reliable measurement tool and was found to be associated with autonomy and critical thinking.

Keywords: nurse, nursing student, competence, autonomy, critical thinking, Rasch analysis

Procedia PDF Downloads 358

Authors: Chia-Chun Wu, Yu-Hsien Sung

Abstract:

This study aims to examine the consistency between students’ predictions and their actual observations on the bullying prevalence rate among different types of high-risk victims, thereby clarifying the reliability of students’ reports on the identification of bullying. A total of 1,732 Taiwanese students (734 males and 998 females) participated in this study. A Rasch model was adopted for data analysis. The results showed that students with “personality or behavioral issues” are more likely to be bullied in schools, based on both students’ predictions and actual observations. Moreover, the results differed significantly between genders and between various educational levels in students’ predictions and their actual observations on the bullying prevalence rate of different types of high-risk victims. To summarize, this study not only suggests that students’ reports on the identification of bullying are accurate and could be a valuable reference in terms of recognizing a bullying incident, but it also argues that more attention should be paid to students’ gender and educational level when taking their perspectives into consideration when it comes to identifying bullying behaviors.

Keywords: school bullying, student, bullying recognition, high-risk victims

Procedia PDF Downloads 55

Authors: Dave Kenneth Tayao Cayado, Carlo P. Magno

Abstract:

The most common source of bias detected are those of gender and socioeconomic status. The present study investigated the Differential Item Functioning (DIF) or item bias between public and private school students in a vocabulary test. Studies on DIF were expanded by using the type of school as a source of bias. There were 200 participants in this study. 100 came from a public secondary school and 100 came from a private secondary school. The vocabulary skills of students were measured using a standardized vocabulary test for grade 7 students. Using DIF, specifically the Rasch-Welch approach, it was found that out of 24 items, 12 were biased for a specific group. The vocabulary skills on the use of slang, idiomatic expression, personification, collocations, and partitive relations were biased for private schools while the use of slang and homonymous words were biased for public school students. The analysis debunked the trend that private school students are outperforming public school students in terms of academic achievement. It was revealed that there are some competencies that private school students are having difficulty and vice versa.

Keywords: differential item functioning, item bias, public school students, private school students, vocabulary

Procedia PDF Downloads 129

Authors: J. Thiele

Abstract:

Teachers working at vocational Colleges are often confronted with the problem, that many students graduated from different schools and therefore each had a different education. Especially in mathematics many students lack fundamentals or had different priorities at their previous schools. Furthermore, these vocational Colleges have to provide Graduations for many different working-fields, with different core themes. The Colleges are interested in measuring the different Education levels of their students and providing assistance for those who need to catch up. The Project mathe-meistern was initiated to remedy this problem at vocational Colleges. For this purpose, online-tests were developed. The aim of these tests is to evaluate basic mathematical abilities of the students. The tests are online Multiple-Choice-Tests with a total of 65 Items. They are accessed online with a unique Transaction-Number (TAN) for each participant. The content is divided in several Categories (Arithmetic, Algebra, Fractions, Geometry, etc.). After each test, the student gets a personalized summary depicting their strengths and weaknesses in mathematical Basics. Teachers can visit a special website to examine the results of their classes or single students. In total 5830 students did participate so far. For standardization and optimization purposes the tests are being evaluated, using the classic and probabilistic Test-Theory regarding Objectivity, Reliability and Validity, annually since 2015. This Paper is about the Optimization process considering the Rasch-scaling and Standardization of the tests. Additionally, current results using standardized tests will be discussed. To achieve this Competence levels and Types of errors of students attending vocational Colleges in Nordrheinwestfalen, Germany, were determined, using descriptive Data and Distractorevaluations.

Keywords: diagnostical tests in mathematics, distractor devaluation, test-optimization, test-theory

Procedia PDF Downloads 97

Authors: Wen-Chung Wang, Xue-Lan Qiu

Abstract:

Ipsative tests have been widely used in vocational and career counseling (e.g., the Jackson Vocational Interest Survey). Pairwise-comparison items are a typical item format of ipsative tests. When the two statements in a pairwise-comparison item measure two different constructs, the item is referred to as a multidimensional pairwise-comparison (MPC) item. A typical MPC item would be: Which activity do you prefer? (A) playing with young children, or (B) working with tools and machines. These two statements aim at the constructs of social interest and investigative interest, respectively. Recently, new item response theory (IRT) models for ipsative tests with MPC items have been developed. Among them, the Rasch ipsative model (RIM) deserves special attention because it has good measurement properties, in which the log-odds of preferring statement A to statement B are defined as a competition between two parts: the sum of a person’s latent trait to which statement A is measuring and statement A’s utility, and the sum of a person’s latent trait to which statement B is measuring and statement B’s utility. The RIM has been extended to polytomous responses, such as preferring statement A strongly, preferring statement A, preferring statement B, and preferring statement B strongly. To promote the new initiatives, in this study we developed computerized adaptive testing algorithms for MFC items and evaluated their performance using simulations and two real tests. Both the RIM and its polytomous extension are multidimensional, which calls for multidimensional computerized adaptive testing (MCAT). A particular issue in MCAT for MPC items is the within-person statement exposure (WPSE); that is, a respondent may keep seeing the same statement (e.g., my life is empty) for many times, which is certainly annoying. In this study, we implemented two methods to control the WPSE rate. In the first control method, items would be frozen when their statements had been administered more than a prespecified times. In the second control method, a random component was added to control the contribution of the information at different stages of MCAT. The second control method was found to outperform the first control method in our simulation studies. In addition, we investigated four item selection methods: (a) random selection (as a baseline), (b) maximum Fisher information method without WPSE control, (c) maximum Fisher information method with the first control method, and (d) maximum Fisher information method with the second control method. These four methods were applied to two real tests: one was a work survey with dichotomous MPC items and the other is a career interests survey with polytomous MPC items. There were three dependent variables: the bias and root mean square error across person measures, and measurement efficiency which was defined as the number of items needed to achieve the same degree of test reliability. Both applications indicated that the proposed MCAT algorithms were successful and there was no loss in measurement proficiency when the control methods were implemented, and among the four methods, the last method performed the best.

Keywords: computerized adaptive testing, ipsative tests, item response theory, pairwise comparison

Procedia PDF Downloads 222

Authors: Alena Kulikova, Leonid Parmaksiz, Ekaterina Orel

Abstract:

Social and emotional skills are considered by modern researchers as predictors of a person's success both in specific areas of activity and in the life of a person as a whole. The popularity of this scientific direction ensures the emergence of a large number of practices aimed at developing and evaluating socio-emotional skills. Assessment of social and emotional development is carried out at the national level, as well as at the level of individual regions and institutions. Despite the fact that many of the already existing social and emotional skills assessment tools are quite convenient and reliable, there are now more and more new technologies and task formats which improve the basic characteristics of the tools. Thus, the goal of the current study is to develop a tool for assessing social and emotional skills such as emotion recognition, emotion regulation, empathy and a culture of self-care. To develop a tool assessing social and emotional skills, Rasch-Gutman scenario-based approach was used. This approach has shown its reliability and merit for measuring various complex constructs: parental involvement; teacher practices that support cultural diversity and equity; willingness to participate in the life of the community after psychiatric rehabilitation; educational motivation and others. To assess emotion recognition, we used a situational judgment task based on OCC (Ortony, Clore, and Collins) emotions theory. The main advantage of these two approaches compare to classical Likert scales is that it reduces social desirability in answers. A field test to check the psychometric properties of the developed instrument was conducted. The instrument was developed for the presidential autonomous non-profit organization “Russia - Land of Opportunity” for nationwide soft skills assessment among higher education students. The sample for the field test consisted of 500 people, students aged from 18 to 25 (mean = 20; standard deviation 1.8), 71% female. 67% of students are only studying and are not currently working and 500 employed adults aged from 26 to 65 (mean = 42.5; SD 9), 57% female. Analysis of the psychometric characteristics of the scales was carried out using the methods of IRT (Item Response Theory). A one-parameter rating scale model RSM (Rating scale model) and Graded Response model (GRM) of the modern testing theory were applied. GRM is a polyatomic extension of the dichotomous two-parameter model of modern testing theory (2PL) based on the cumulative logit function for modeling the probability of a correct answer. The validity of the developed scales was assessed using correlation analysis and MTMM (multitrait-multimethod matrix). The developed instrument showed good psychometric quality and can be used by HR specialists or educational management. The detailed results of a psychometric study of the quality of the instrument, including the functioning of the tasks of each scale, will be presented. Also, the results of the validity study by MTMM analysis will be discussed.

Keywords: social and emotional skills, psychometrics, MTMM, IRT

Procedia PDF Downloads 42

Authors: Stephanie J. Sedberry, William J. Gerace, Ian D. Beatty, Michael J. Kane

Abstract:

Increasing the number of US university students who attain degrees in STEM and enter the STEM workforce is a national priority. Demographic groups vary in their rates of participation in STEM, and the US produces just 10% of the world’s science and engineering degrees (2014 figures). To address these gaps, we have developed and tested a practical, 30-minute, single-session classroom-based intervention to improve students’ self-efficacy and academic performance in University STEM courses. Self-efficacy is a psychosocial construct that strongly correlates with academic success. Self-efficacy is a construct that is internal and relates to the social, emotional, and psychological aspects of student motivation and performance. A compelling body of research demonstrates that university students’ self-efficacy beliefs are strongly related to their selection of STEM as a major, aspirations for STEM-related careers, and persistence in science. The development of an intervention to increase students’ self-efficacy is motivated by research showing that short, social-psychological interventions in education can lead to large gains in student achievement. Our intervention addresses STEM self-efficacy via two strong, but previously separate, lines of research into attitudinal/affect variables that influence student success. The first is ‘attributional retraining,’ in which students learn to attribute their successes and failures to internal rather than external factors. The second is ‘mindset’ about fixed vs. growable intelligence, in which students learn that the brain remains plastic throughout life and that they can, with conscious effort and attention to thinking skills and strategies, become smarter. Extant interventions for both of these constructs have significantly increased academic performance in the classroom. We developed a 34-item questionnaire (Likert scale) to measure STEM Self-efficacy, Perceived Academic Control, and Growth Mindset in a University STEM context, and validated it with exploratory factor analysis, Rasch analysis, and multi-trait multi-method comparison to coded interviews. Four iterations of our 42-week research protocol were conducted across two academic years (2017-2018) at three different Universities in North Carolina, USA (UNC-G, NC A&T SU, and NCSU) with varied student demographics. We utilized a quasi-experimental prospective multiple-group time series research design with both experimental and control groups, and we are employing linear modeling to estimate the impact of the intervention on Self-Efficacy,wth-Mindset, Perceived Academic Control, and final course grades (performance measure). Preliminary results indicate statistically significant effects of treatment vs. control on Self-Efficacy, Growth-Mindset, Perceived Academic Control. Analyses are ongoing and final results pending. This intervention may have the potential to increase student success in the STEM classroom—and ownership of that success—to continue in a STEM career. Additionally, we have learned a great deal about the complex components and dynamics of self-efficacy, their link to performance, and the ways they can be impacted to improve students’ academic performance.

Keywords: academic performance, affect variables, growth mindset, intervention, perceived academic control, psycho-social variables, self-efficacy, STEM, university classrooms

Procedia PDF Downloads 103