Search results for: Kinect V2

Authors: Amin Amini

Abstract:

A Kinect sensor has been utilized as a cheap and accurate alternative to 3D laser scanners and electronic distance measurement (EDM) systems. This research presents an inexpensive and easy-to-setup system that utilizes the Microsoft Kinect v2 sensor as a surveying and measurement tool and investigates the possibility of using such a device as a replacement for conventional theodolite systems. The system was tested in an indoor environment where its accuracy in distance and angle measurements was tested using virtual markers in a 3D Cartesian environment. The system has shown an average accuracy of 97.94 % in measuring distances and 99.11 % and 98.84 % accuracy for area and perimeter, respectively, within the Kinect’s surveying range of 1.5 to 6 meters. The research also tested the system competency for relative angle determination between two objects.

Keywords: kinect v2, 3D measurement, depth map, ToF

Procedia PDF Downloads 66

Authors: Justin Tang

Abstract:

Balance tests are frequently used to diagnose concussions on the sidelines of sporting events. Manual scoring, however, is labor intensive and subjective, and many concussions go undetected. This study institutes a novel approach to conducting the Balance Error Scoring System (BESS) more quantitatively using Microsoft’s gaming system Kinect, which uses a contactless sensor and several cameras to receive data and estimate body limb positions. Using a machine learning approach, Visual Gesture Builder, and a deterministic approach, MATLAB, we tested whether the Kinect can differentiate between “correct” and erroneous stances of the BESS. We created the two separate solutions by recording test videos to teach the Kinect correct stances and by developing a code using Java. Twenty-two subjects were asked to perform a series of BESS tests while the Kinect was collecting data. The Kinect recorded the subjects and mapped key joints onto their bodies to obtain angles and measurements that are interpreted by the software. Through VGB and MATLAB, the videos are analyzed to enumerate the number of errors committed during testing. The resulting statistics demonstrate a high correlation between manual scoring and the Kinect approaches, indicating the viability of the use of remote tracking devices in conducting concussion tests.

Keywords: automated, concussion detection, contactless sensors, microsoft kinect

Procedia PDF Downloads 316

Authors: Bernstein Ran, Shafir Tal, Tsachor Rachelle, Studd Karen, Schuster Assaf

Abstract:

Laban Movement Analysis (LMA), developed in the dance community over the past seventy years, is an effective method for observing, describing, notating, and interpreting human movement to enhance communication and expression in everyday and professional life. Many applications that use motion capture data might be significantly leveraged if the Laban qualities will be recognized automatically. This paper presents an automated recognition method of Laban qualities from motion capture skeletal recordings and it is demonstrated on the output of Microsoft’s Kinect V2 sensor.

Keywords: Laban movement analysis, multitask learning, Kinect sensor, machine learning

Procedia PDF Downloads 339

Authors: Bo Wan, Xiu Wen, Lingling An, Xiaoling Ding

Abstract:

The paper describes a Chinese shadow play animation system based on Kinect. Users, without any professional training, can personally manipulate the shadow characters to finish a shadow play performance by their body actions and get a shadow play video through giving the record command to our system if they want. In our system, Kinect is responsible for capturing human movement and voice commands data. Gesture recognition module is used to control the change of the shadow play scenes. After packaging the data from Kinect and the recognition result from gesture recognition module, VRPN transmits them to the server-side. At last, the server-side uses the information to control the motion of shadow characters and video recording. This system not only achieves human-computer interaction, but also realizes the interaction between people. It brings an entertaining experience to users and easy to operate for all ages. Even more important is that the application background of Chinese shadow play embodies the protection of the art of shadow play animation.

Keywords: hadow play animation, Kinect, gesture recognition, VRPN, HCI

Procedia PDF Downloads 400

Authors: Rania Ahmed Kadry Abdel Gawad Birry, Mohamed El-Habrouk

Abstract:

This paper presents a brief survey of the techniques used for sign language recognition along with the types of sensors used to perform the task. It presents a modified method for identification of an isolated sign language gesture using Microsoft Kinect with the OpenNI framework. It presents the way of extracting robust features from the depth image provided by Microsoft Kinect and the OpenNI interface and to use them in creating a robust and accurate gesture recognition system, for the purpose of ASL translation. The Prime Sense’s Natural Interaction Technology for End-user - NITE™ - was also used in the C++ implementation of the system. The algorithm presents a simple finger counting algorithm for static signs as well as directional Finite State Machine (FSM) description of the hand motion in order to help in translating a sign language gesture. This includes both letters and numbers performed by a user, which in-turn may be used as an input for voice pronunciation systems.

Keywords: American sign language, finger counting, hand tracking, Microsoft Kinect

Procedia PDF Downloads 294

Authors: Shih-Wen Hsiao, Yi-Cheng Tsao

Abstract:

In the field of reverse engineering and creative industries, applying 3D scanning process to obtain geometric forms of the objects is a mature and common technique. For instance, organic objects such as faces and nonorganic objects such as products could be scanned to acquire the geometric information for further application. However, although the data resolution of 3D scanning device is increasing and there are more and more abundant complementary applications, the penetration rate of 3D scanning for the public is still limited by the relative high price of the devices. On the other hand, Kinect, released by Microsoft, is known for its powerful functions, considerably low price, and complete technology and database support. Therefore, related studies can be done with the applying of Kinect under acceptable cost and data precision. Due to the fact that Kinect utilizes optical mechanism to extracting depth information, limitations are found due to the reason of the straight path of the light. Thus, various angles are required sequentially to obtain the complete 3D information of the object when applying a single Kinect for 3D scanning. The integration process which combines the 3D data from different angles by certain algorithms is also required. This sequential scanning process costs much time and the complex integration process often encounter some technical problems. Therefore, this paper aimed to apply multiple Kinects simultaneously on the field of developing a rapid 3D mannequin scan platform and proposed suggestions on the number and angles of Kinects. In the content, a method of establishing the coordination based on the relation between mannequin and the specifications of Kinect is proposed, and a suggestion of angles and number of Kinects is also described. An experiment of applying multiple Kinect on the scanning of 3D mannequin is constructed by Microsoft API, and the results show that the time required for scanning and technical threshold can be reduced in the industries of fashion and garment design.

Keywords: 3D scan, depth sensor, fashion and garment design, mannequin, multiple Kinect sensor

Procedia PDF Downloads 366

Authors: Shih-Wen Hsiao, Rong-Qi Chen

Abstract:

In the field of fashion design, 3D Mannequin is a kind of assisting tool which could rapidly realize the design concepts. While the concept of 3D Mannequin is applied to the computer added fashion design, it will connect with the development and the application of design platform and system. Thus, the situation mentioned above revealed a truth that it is very critical to develop a module of 3D Mannequin which would correspond with the necessity of fashion design. This research proposes a concrete plan that developing and constructing a system of 3D Mannequin with Kinect. In the content, ergonomic measurements of objective human features could be attained real-time through the implement with depth camera of Kinect, and then the mesh morphing can be implemented through transformed the locations of the control-points on the model by inputting those ergonomic data to get an exclusive 3D mannequin model. In the proposed methodology, after the scanned points from the Kinect are revised for accuracy and smoothening, a complete human feature would be reconstructed by the ICP algorithm with the method of image processing. Also, the objective human feature could be recognized to analyze and get real measurements. Furthermore, the data of ergonomic measurements could be applied to shape morphing for the division of 3D Mannequin reconstructed by feature curves. Due to a standardized and customer-oriented 3D Mannequin would be generated by the implement of subdivision, the research could be applied to the fashion design or the presentation and display of 3D virtual clothes. In order to examine the practicality of research structure, a system of 3D Mannequin would be constructed with JAVA program in this study. Through the revision of experiments the practicability-contained research result would come out.

Keywords: 3D mannequin, kinect scanner, interactive closest point, shape morphing, subdivision

Procedia PDF Downloads 304

Authors: Reem Alshiha, Tanzila Saba

Abstract:

Physical rehabilitation is the process of helping people to recover and be able to go back to their former activities that have been delayed due to external factors such as car accidents, old age and victims of strokes (chronic diseases and accidents, and those related to sport activities).The cost of hiring a personal nurse or driving the patient to and from the hospital could be costly and time-consuming. Also, there are other factors to take into account such as forgetfulness, boredom and lack of motivation. In order to solve this dilemma, some experts came up with rehabilitation software to be used with Microsoft Kinect to help the patients and their families for in-home rehabilitation. In home rehabilitation software is becoming more and more popular, since it is more convenient for all parties affiliated with the patient. In contrast to the other costly market-based systems that have no portability, Microsoft’s Kinect is a portable motion sensor that reads body movements and interprets it. New software development has made rehabilitation games available to be used at home for the convenience of the patient. The game will benefit its users (rehabilitation patients) in saving time and money. There are many software's that are used with the Kinect for rehabilitation, but the software that is chosen in this research is Kinectotherapy. Kinectotherapy software is used for rehabilitation patients in Riyadh clinics to test its acceptance by patients and their physicians. In this study, we used Kinect because it was affordable, portable and easy to access in contrast to expensive market-based motion sensors. This paper explores the importance of in-home rehabilitation by using Kinect with Kinectotherapy software. The software targets both upper and lower limbs, but in this research, the main focus is on upper-limb functionality. However, the in-home rehabilitation is applicable to be used by all patients with motor disability, since the patient must have some self-reliance. The targeted subjects are patients with minor motor impairment that are somewhat independent in their mobility. The presented work is the first to consider the implementation of in-home rehabilitation with real-time feedback to the patient and physician. This research proposes the implementation of in-home rehabilitation in Riyadh, Saudi Arabia. The findings show that most of the patients are interested and motivated in using the in-home rehabilitation system in the future. The main value of the software application is due to these factors: improve patient engagement through stimulating rehabilitation, be a low cost rehabilitation tool and reduce the need for expensive one-to-one clinical contact. Rehabilitation is a crucial treatment that can improve the quality of life and confidence of the patient as well as their self-esteem.

Keywords: x-box, rehabilitation, physical therapy, rehabilitation software, kinect

Procedia PDF Downloads 340

Authors: Ankita Wadhawan, Parteek Kumar, Umesh Kumar

Abstract:

Gesture is any body movement that expresses some attitude or any sentiment. Gestures as a sign language are used by deaf people for conveying messages which helps in eliminating the communication barrier between deaf people and normal persons. Nowadays, everybody is using mobile phone and computer as a very important gadget in their life. But there are some physically challenged people who are blind/deaf and the use of mobile phone or computer like device is very difficult for them. So, there is an immense need of a system which works on body gesture or sign language as input. In this research, Microsoft Kinect Sensor, SDK V2 and Hidden Markov Toolkit (HTK) are used to recognize the object, motion of object and human body joints through Touch less NUI (Natural User Interface) in real-time. The depth data collected from Microsoft Kinect has been used to recognize gestures of Indian Sign Language (ISL). The recorded clips are analyzed using depth, IR and skeletal data at different angles and positions. The proposed system has an average accuracy of 85%. The developed Touch less NUI provides an interface to recognize gestures and controls the cursor and click operation in computer just by waving hand gesture. This research will help deaf people to make use of mobile phones, computers and socialize among other persons in the society.

Keywords: gesture recognition, Indian sign language, Microsoft Kinect, natural user interface, sign language

Procedia PDF Downloads 303

Authors: Elham Saraee, Saurabh Singh, Margrit Betke

Abstract:

Evaluation of a person’s posture while exercising is important in physical therapy. During a therapy session, a physical therapist or a monitoring system must assure that the person is performing an exercise correctly to achieve the desired therapeutic effect. In this work, we introduce a system called POSTURECHECK for exercise assessment in physical therapy. POSTURECHECK assesses the posture of a person who is exercising with the Proficio robotic arm while being recorded by the Microsoft Kinect interface. POSTURECHECK extracts unique features from the person’s upper body during the exercise, and classifies the sequence of postures as correct or incorrect using Bayesian estimation and majority voting. If POSTURECHECK recognizes an incorrect posture, it specifies what the user can do to correct it. The result of our experiment shows that POSTURECHECK is capable of recognizing the incorrect postures in real time while the user is performing an exercise.

Keywords: Bayesian estimation, majority voting, Microsoft Kinect, PostureCheck, Proficio robotic arm, upper body physical therapy

Procedia PDF Downloads 279

Authors: M. Owlia, M. H. Azarsa, M. Khabbazan, A. Mirbagheri

Abstract:

Proprioceptive neuromuscular facilitation exercises are a series of stretching techniques that are commonly used in rehabilitation and exercise therapy. Assessment of these exercises for true maneuvering requires extensive experience in this field and could not be down with patients themselves. In this paper, we developed software that uses Microsoft Kinect sensor, a spherical color marker, and real-time image processing methods to evaluate patient’s performance in generating true patterns of movements. The software also provides the patient with a visual feedback by showing his/her avatar in a Virtual Reality environment along with the correct path of moving hand, wrist and marker. Primary results during PNF exercise therapy of a patient in a room environment shows the ability of the system to identify any deviation of maneuvering path and direction of the hand from the one that has been performed by an expert physician.

Keywords: image processing, Microsoft Kinect, proprioceptive neuromuscular facilitation, upper extremities assessment, virtual reality

Procedia PDF Downloads 273

Authors: M. Jazouli, S. Elhoufi, A. Majda, A. Zarghili, R. Aalouane

Abstract:

Autism spectrum disorder is a complex developmental disability. It is defined by a certain set of behaviors. Persons with Autism Spectrum Disorders (ASD) frequently engage in stereotyped and repetitive motor movements. The objective of this article is to propose a method to automatically detect this unusual behavior. Our study provides a clinical tool which facilitates for doctors the diagnosis of ASD. We focus on automatic identification of five repetitive gestures among autistic children in real time: body rocking, hand flapping, fingers flapping, hand on the face and hands behind back. In this paper, we present a gesture recognition system for children with autism, which consists of three modules: model-based movement tracking, feature extraction, and gesture recognition using artificial neural network (ANN). The first one uses the Microsoft Kinect sensor, the second one chooses points of interest from the 3D skeleton to characterize the gestures, and the last one proposes a neural connectionist model to perform the supervised classification of data. The experimental results show that our system can achieve above 93.3% recognition rate.

Keywords: ASD, artificial neural network, kinect, stereotypical motor movements

Procedia PDF Downloads 304

Authors: Maryam Savari, Mohamad Nizam Ayub, Ainuddin Wahid Abdul Wahab

Abstract:

Game-based Learning (GBL) is an alternative that provides learners with an opportunity to experience a volatile environment in a safe and secure place. A volatile environment requires a different technique to facilitate learning and prevent injury and other hazards. Subjects involving elasticity are always considered hazardous and can cause injuries,for instance a bouncing ball. Elasticity is a topic that necessitates hands-on practicality for learners to experience the effects of elastic objects. In this paper the scope is to investigate the natural interaction between learners and elastic objects in a safe environment using GBL. During interaction, the potentials of natural contact in the process of learning were explored and gestures exhibited during the learning process were identified. GBL was developed using Kinect technology to teach elasticity to primary school children aged 7 to 12. The system detects body gestures and defines the meanings of motions exhibited during the learning process. The qualitative approach was deployed to constantly monitor the interaction between the student and the system. Based on the results, it was found that Natural Interaction GBL (Ni-GBL) is engaging for students to learn, making their learning experience more active and joyful.

Keywords: elasticity, Game-Based Learning (GBL), kinect technology, natural interaction

Procedia PDF Downloads 482

Authors: Maysam Shahsavari, Seyed Jamalaldin Haddadi

Abstract:

There are several methods to localize a mobile robot such as relative, absolute and probabilistic. In this paper, particle filter due to its simple implementation and the fact that it does not need to know to the starting position will be used. This method estimates the position of the mobile robot using a probabilistic distribution, relying on a known map of the environment instead of predicting it. Afterwards, it updates this estimation by reading input sensors and control commands. To receive information from the surrounding world, distance to obstacles, for example, a Kinect is used which is much cheaper than a laser range finder. Finally, after explaining the Adaptive Particle Filter method and its implementation in detail, we will compare this method with the dead reckoning method and show that this method is much more suitable for situations in which we have a map of the environment.

Keywords: particle filter, localization, methods, odometry, kinect

Procedia PDF Downloads 264

Authors: Maha Jazouli

Abstract:

Suicide is one of the most important causes of death in the prison environment, both in Canada and internationally. Rates of attempts of suicide and self-harm have been on the rise in recent years, with hangings being the most frequent method resorted to. The objective of this article is to propose a method to automatically detect in real time suicidal behaviors. We present a gesture recognition system that consists of three modules: model-based movement tracking, feature extraction, and gesture recognition using machine learning algorithms (MLA). Our proposed system gives us satisfactory results. This smart video surveillance system can help assist staff responsible for the safety and health of inmates by alerting them when suicidal behavior is detected, which helps reduce mortality rates and save lives.

Keywords: suicide detection, Kinect azure, RGB-D camera, SVM, machine learning, gesture recognition

Procedia PDF Downloads 187

Authors: Kathia Melbouci, Sylvie Naudet Collette, Vincent Gay-Bellile, Omar Ait-Aider, Michel Dhome

Abstract:

In this paper, we handle the problem of real time localization and mapping in indoor environment assisted by a partial prior 3D model, using an RGBD sensor. The proposed solution relies on a feature-based RGBD SLAM algorithm to localize the camera and update the 3D map of the scene. To improve the accuracy and the robustness of the localization, we propose to combine in a local bundle adjustment process, geometric information provided by a prior coarse 3D model of the scene (e.g. generated from the 2D floor plan of the building) along with RGBD data from a Kinect camera. The proposed approach is evaluated on a public benchmark dataset as well as on real scene acquired by a Kinect sensor.

Keywords: SLAM, global localization, 3D sensor, bundle adjustment, 3D model

Procedia PDF Downloads 410

Authors: Sanjana S. Mallya, Roshan Arvind Sivakumar

Abstract:

Orthotics is the branch of medicine that deals with the provision and use of artificial casts or braces to alter the biomechanical structure of the limb and provide support for the limb. Custom-made orthoses provide more comfort and can correct issues better than those available over-the-counter. However, they are expensive and require intricate modelling of the limb. Traditional methods of modelling involve creating a plaster of Paris mould of the limb. Lately, CAD/CAM and 3D printing processes have improved the accuracy and reduced the production time. Ordinarily, digital cameras are used to capture the features of the limb from different views to create a 3D model. We propose a system to model the limb using Microsoft Kinect2 sensor. The Kinect can capture RGB and depth frames simultaneously up to 30 fps with sufficient accuracy. The region of interest is captured from three views, each shifted by 90 degrees. The RGB and depth data are fused into a single RGB-D frame. The resolution of the RGB frame is 1920px x 1080px while the resolution of the Depth frame is 512px x 424px. As the resolution of the frames is not equal, RGB pixels are mapped onto the Depth pixels to make sure data is not lost even if the resolution is lower. The resulting RGB-D frames are collected and using the depth coordinates, a three dimensional point cloud is generated for each view of the Kinect sensor. A common reference system was developed to merge the individual point clouds from the Kinect sensors. The reference system consisted of 8 coloured cubes, connected by rods to form a skeleton-cube with the coloured cubes at the corners. For each Kinect, the region of interest is the square formed by the centres of the four cubes facing the Kinect. The point clouds are merged by considering one of the cubes as the origin of a reference system. Depending on the relative distance from each cube, the three dimensional coordinate points from each point cloud is aligned to the reference frame to give a complete point cloud. The RGB data is used to correct for any errors in depth data for the point cloud. A triangular mesh is generated from the point cloud by applying Delaunay triangulation which generates the rough surface of the limb. This technique forms an approximation of the surface of the limb. The mesh is smoothened to obtain a smooth outer layer to give an accurate model of the limb. The model of the limb is used as a base for designing the custom orthotic brace or cast. It is transferred to a CAD/CAM design file to design of the brace above the surface of the limb. The proposed system would be more cost effective than current systems that use MRI or CT scans for generating 3D models and would be quicker than using traditional plaster of Paris cast modelling and the overall setup time is also low. Preliminary results indicate that the accuracy of the Kinect2 is satisfactory to perform modelling.

Keywords: 3d scanning, mesh generation, Microsoft kinect, orthotics, registration

Procedia PDF Downloads 189

Authors: Shyam Sajan, Kamal Bijlani

Abstract:

The widespread effects of digital media help people to explore the world more and get entertained with no effort. People became fond of these kind of sedentary life style. The increase in sedentary time and a decrease in physical activities has negative impacts on human health. Even though the addiction to video games has been exploited in exergames, to make people exercise and enjoy game challenges, the contribution is restricted only to physical wellness. This paper proposes creation and implementation of a game with the help of digital media in a virtual environment. The game is designed by collaborating ideas from neural linguistic programming and Stroop effect that can also be used to identify a person’s mental state, to improve concentration and to eliminate various phobias. The multiplayer game is played in a virtual environment created with Kinect sensor, to make the game more motivating and interactive.

Keywords: exergaming, Kinect Sensor, Neural Linguistic Programming, Stroop Effect

Procedia PDF Downloads 435

Authors: Vijay Kumar Kolagani, Yingcai Xiao

Abstract:

Adaptation of Natural User Interface (NUI) has been slow and limited. NUI devices like Microsoft’s Kinect and Ultraleap’s Leap Motion can only interact with a handful applications that were specifically designed and implemented for them. A NUI device just can’t be used to directly control millions of applications that are not designed to take NUI input. This is in the similar situation like the adaptation of color TVs. At the early days of color TV, the broadcasting format was in RGB, which was not viewable by blackand-white TVs. TV broadcasters were reluctant to produce color programs due to limited viewership. TV viewers were reluctant to buy color TVs because there were limited programs to watch. Color TV’s breakthrough moment came after the adaptation of NTSC standard which allowed color broadcasts to be compatible with the millions of existing black-and-white TVs. This research presents a framework to use NUI devices to control existing non-NUI applications without reprogramming them. The methodology is to create an adapter to convert input from NUI devices into input compatible with that generated by CLI (Command Line Input) and GUI (Graphical User Interface) devices. The CLI/GUI compatible input is then sent to the active application through the operating system just like any input from a CLI/GUI device to control the non-NUI program that the user is controlling. A sample adapter has been created to convert input from Kinect to keyboard strokes, so one can use the input from Kinect to control any applications that take keyboard input, such as Microsoft’s PowerPoint. When the users use the adapter to control their PowerPoint presentations, they can free themselves from standing behind a computer to use its keyboard and can roam around in front of the audience to use hand gestures to control the PowerPoint. It is hopeful such adapters can accelerate the adaptation of NUI devices.

Keywords: command line input, graphical user interface, human computer interaction, natural user interface, NUI adapter

Procedia PDF Downloads 12

Authors: Sneha Noble, Rahul Krishnan, Uma G., D. K. Vijaykumar

Abstract:

Breast-Cancer related Lymphedema is a major problem that affects many women. Lymphedema is the swelling that generally occurs in the arms or legs caused by the removal of or damage to lymph nodes as a part of cancer treatment. Treating it at the earliest possible stage is the best way to manage the condition and prevent it from leading to pain, recurrent infection, reduced mobility, and impaired function. So, this project aims to focus on the multi-modal approaches to identify the risks of Lymphedema in post-surgical oncology patients and prevent it at the earliest. The Kinect IR Sensor is utilized to capture the images of the body and after image processing techniques, the region of interest is obtained. Then, performing the voxelization method will provide volume measurements in pre-operative and post-operative periods in patients. The formation of a mathematical model will help in the comparison of values. Clinical pathological data of patients will be investigated to assess the factors responsible for the development of lymphedema and its risks.

Keywords: Kinect IR sensor, Lymphedema, voxelization, lymph nodes

Procedia PDF Downloads 136

Authors: Rodrigo C. Belleza, Jr., Roselyn A. Maaño, Karl Patrick E. Camota, Darwin Kim Q. Bulawan

Abstract:

Eyes are an essential and conspicuous organ of the human body. Human eyes are outward and inward portals of the body that allows to see the outside world and provides glimpses into ones inner thoughts and feelings. Inevitable blindness and visual impairments may result from eye-related disease, trauma, or congenital or degenerative conditions that cannot be corrected by conventional means. The study emphasizes innovative tools that will serve as an aid to the blind and visually impaired (VI) individuals. The researchers fabricated a prototype that utilizes the Microsoft Kinect for Windows and Arduino microcontroller board. The prototype facilitates advanced gesture recognition, voice recognition, obstacle detection and indoor environment navigation. Open Computer Vision (OpenCV) performs image analysis, and gesture tracking to transform Kinect data to the desired output. A computer vision technology device provides greater accessibility for those with vision impairments.

Keywords: algorithms, blind, computer vision, embedded systems, image analysis

Procedia PDF Downloads 316

Authors: V. Ramu Reddy, Parijat Deshpande, Ranjan Dasgupta

Abstract:

This paper presents different methodology for an acoustic source sensing and localization in an unknown environment. The developed methodology includes an acoustic based sensing and localization system, a converging target localization based on the recursive direction of arrival (DOA) error minimization, and a regressive obstacle avoidance function. Our method is able to augment the existing proven localization techniques and improve results incrementally by utilizing robot locomotion and is capable of converging to a position estimate with greater accuracy using fewer measurements. The results also evinced the DOA error minimization at each iteration, improvement in time for reaching the destination and the efficiency of this target localization method as gradually converging to the real target position. Initially, the system is tested using Kinect mounted on turntable with DOA markings which serve as a ground truth and then our approach is validated using a FireBird VI (FBVI) mobile robot on which Kinect is used to obtain bearing information.

Keywords: acoustic source localization, acoustic sensing, recursive direction of arrival, robot locomotion

Procedia PDF Downloads 490

Authors: Carlos Gordon, Patricio Encalada, Henry Lema, Diego Leon, Dennis Chicaiza

Abstract:

The following work presents a proposal of autonomous navigation of mobile robots implemented in an omnidirectional robot Kuka Youbot. We have been able to perform the integration of robotic operative system (ROS) and machine learning algorithms. ROS mainly provides two distributions; ROS hydro and ROS Kinect. ROS hydro allows managing the nodes of odometry, kinematics, and path planning with statistical and probabilistic, global and local algorithms based on Adaptive Monte Carlo Localization (AMCL) and Dijkstra. Meanwhile, ROS Kinect is responsible for the detection block of dynamic objects which can be in the points of the planned trajectory obstructing the path of Kuka Youbot. The detection is managed by artificial vision module under a trained neural network based on the single shot multibox detector system (SSD), where the main dynamic objects for detection are human beings and domestic animals among other objects. When the objects are detected, the system modifies the trajectory or wait for the decision of the dynamic obstacle. Finally, the obstacles are skipped from the planned trajectory, and the Kuka Youbot can reach its goal thanks to the machine learning algorithms.

Keywords: autonomous navigation, machine learning, path planning, robotic operative system, open source computer vision library

Procedia PDF Downloads 175

Authors: Rohit Semwal, Shivam Arora, Saurav, Sangita Roy

Abstract:

This work proposes a supervised framework with deep convolutional neural networks (CNNs) for vision-based sign language recognition of static gestures. Our approach addresses the acquisition and segmentation of correct inputs for the CNN-based classifier. Microsoft Kinect™ sensor, despite complex environmental conditions, can track hands efficiently. Skin Colour based segmentation is applied on cropped images of hands in different poses, used to depict different sign language gestures. The segmented hand images are used as an input for our classifier. The CNN classifier proposed in the paper is able to classify the input images with a high degree of accuracy. The system was trained and tested on 39 static sign language gestures, including 26 letters of the alphabet and 13 commonly used words. This paper includes a problem definition for building the proposed system, which acts as a sign language translator between deaf/mute and the rest of the society. It is then followed by a focus on reviewing existing knowledge in the area and work done by other researchers. It also describes the working principles behind different components of CNNs in brief. The architecture and system design specifications of the proposed system are discussed in the subsequent sections of the paper to give the reader a clear picture of the system in terms of the capability required. The design then gives the top-level details of how the proposed system meets the requirements.

Keywords: sign language, CNN, HCI, segmentation

Procedia PDF Downloads 156

Authors: Haruya Tamaki, Tsugunosuke Sakai, Ryuichi Yoshida, Ryohei Egusa, Shigenori Inagaki, Etsuji Yamaguchi, Fusako Kusunoki, Miki Namatame, Masanori Sugimoto, Hiroshi Mizoguchi

Abstract:

Art appreciation is important as part of children education. Art appreciation can enrich sensibility and creativity. To enrich sensibility and creativity, the children have to learning knowledge of picture such as social and historical backgrounds and author intention. High learning effect can acquire by actively learning. In short, it is important that encourage learning of the knowledge about pictures actively. It is necessary that children feel like interest to encourage learning of the knowledge about pictures actively. In a general art museum, comments on pictures are done through writing. Thus, we expect that this method cannot arouse the interest of the children in pictures, because children feel like boring. In brief, learning about the picture information is difficult. Therefore, we are developing an art-appreciation support system that will encourage learning of the knowledge about pictures actively by children feel like interest. This system uses that Interacting with Pictures to learning of the knowledge about pictures. To Interacting with Pictures, children have to utterance by themselves. We expect that will encourage learning of the knowledge about pictures actively by Interacting with Pictures. To more actively learning, children can choose who talking with by information that location and movement of the children. This system must be able to acquire real-time knowledge of the location, movement, and voice of the children. We utilize the Microsoft’s Kinect v2 sensor and its library, namely, Kinect for Windows SDK and Speech Platform SDK v11 for this purpose. By using these sensor and library, we can determine the location, movement, and voice of the children. As the first step of this system, we developed ukiyo-e game that use ukiyo-e to appreciation object. Ukiyo-e is a traditional Japanese graphic art that has influenced the western society. Therefore, we believe that the ukiyo-e game will be appreciated. In this study, we applied talking to pictures to learn information about the pictures because we believe that learning information about the pictures by talking to the pictures is more interesting than commenting on the pictures using only texts. However, we cannot confirm if talking to the pictures is more interesting than commenting using texts only. Thus, we evaluated through EDA measurement whether the user develops an interest in the pictures while talking to them using voice recognition or by commenting on the pictures using texts only. Hence, we evaluated that children have interest to picture while talking to them using voice recognition through EDA measurement. In addition, we quantitatively evaluate that enjoyed this game or not and learning information about the pictures for primary schoolchildren. In this paper, we summarize these two evaluation results.

Keywords: actively learning, art appreciation, EDA, Kinect V2

Procedia PDF Downloads 284

Authors: Alexandros Lioulemes, Michail Theofanidis, Varun Kanal, Konstantinos Tsiakas, Maher Abujelala, Chris Collander, William B. Townsend, Angie Boisselle, Fillia Makedon

Abstract:

This paper presents a home-based robot-rehabilitation instrument, called ”MAGNI Dynamics”, that utilized a vision-based kinematic/dynamic module and an adaptive haptic feedback controller. The system is expected to provide personalized rehabilitation by adjusting its resistive and supportive behavior according to a fuzzy intelligence controller that acts as an inference system, which correlates the user’s performance to different stiffness factors. The vision module uses the Kinect’s skeletal tracking to monitor the user’s effort in an unobtrusive and safe way, by estimating the torque that affects the user’s arm. The system’s torque estimations are justified by capturing electromyographic data from primitive hand motions (Shoulder Abduction and Shoulder Forward Flexion). Moreover, we present and analyze how the Barrett WAM generates a force-field with a haptic controller to support or challenge the users. Experiments show that by shifting the proportional value, that corresponds to different stiffness factors of the haptic path, can potentially help the user to improve his/her motor skills. Finally, potential areas for future research are discussed, that address how a rehabilitation robotic framework may include multisensing data, to improve the user’s recovery process.

Keywords: human-robot interaction, kinect, kinematics, dynamics, haptic control, rehabilitation robotics, artificial intelligence

Procedia PDF Downloads 327

Authors: Luo Jiaoyang, Yu Hongyang

Abstract:

In this paper, we explore a method that can be used in the working scene of intelligent industrial robots to confirm the identity information of operators to ensure that the robot executes instructions in a sufficiently safe environment. This approach uses three information modalities, namely visible light, depth, and sound. We explored a variety of fusion modes for the three modalities and finally used the joint feature learning method to improve the performance of the model in the case of noise compared with the single-modal case, making the maximum noise in the experiment. It can also maintain an accuracy rate of more than 90%.

Keywords: multimodal, kinect, machine learning, distance image

Procedia PDF Downloads 78

Authors: Do-Jin Jang, Sung-Ah Kim

Abstract:

In designing a kinetic façade, it is hard for the designer to make digital models due to its complex geometry with motion. This paper aims to present a methodology of converting a point cloud of a physical model into a single digital model with a certain topology and motion. The method uses a Microsoft Kinect sensor, and color markers were defined and applied to three paper folding-inspired designs. Although the resulted digital model cannot represent the whole folding range of the physical model, the method supports the designer to conduct a performance-oriented design process with the rough physical model in the reduced folding range.

Keywords: design media, kinetic facades, tangible user interface, 3D scanning

Procedia PDF Downloads 412

Authors: Daniel Vargas-Herrera, Ivette Caldelas, Fernando Brambila-Paz, Rodrigo Montufar-Chaveznava

Abstract:

In this paper, we present the advances corresponding to the implementation of a set of educational materials based on video games technologies. Essentially these materials correspond to projects developed and under development as bachelor thesis of some Computer Engineering students of the Engineering School. All materials are based on the Unity SDK; integrating some devices such as kinect, leap motion, oculus rift, data gloves and Google cardboard. In detail, we present a virtual reality application for neurosciences students (suitable for neural rehabilitation), and virtual scenes for the Google cardboard, which will be used by the psychology students for phobias treatment. The objective is these materials will be located at a server to be available for all students, in the classroom or in the cloud, considering the use of smartphones has been widely extended between students.

Keywords: virtual reality, interactive technologies, video games, educational materials

Procedia PDF Downloads 655

Authors: Somar Boubou, Tatsuo Narikiyo, Michihiro Kawanishi

Abstract:

In this work, we address the problem of 3D object recognition with depth sensors such as Kinect or Structure sensor. Compared with traditional approaches based on local descriptors, which depends on local information around the object key points, we propose a global features based descriptor. Proposed descriptor, which we name as Differential Histogram of Normal Vectors (DHONV), is designed particularly to capture the surface geometric characteristics of the 3D objects represented by depth images. We describe the 3D surface of an object in each frame using a 2D spatial histogram capturing the normalized distribution of differential angles of the surface normal vectors. The object recognition experiments on the benchmark RGB-D object dataset and a self-collected dataset show that our proposed descriptor outperforms two others descriptors based on spin-images and histogram of normal vectors with linear-SVM classifier.

Keywords: vision in control, robotics, histogram, differential histogram of normal vectors

Procedia PDF Downloads 277