C Copyright 1999 By Joseph J Laviola Jr -Books Pdf

c Copyright 1999 by Joseph J LaViola Jr
07 Apr 2020 | 16 views | 0 downloads | 137 Pages | 1.39 MB

Share Pdf : C Copyright 1999 By Joseph J Laviola Jr

Download and Preview : C Copyright 1999 By Joseph J Laviola Jr


Report CopyRight/DMCA Form For : C Copyright 1999 By Joseph J Laviola Jr



Transcription

c Copyright 1999 by Joseph J LaViola Jr, This dissertation by Joseph J LaViola Jr is accepted in its present form by. the Department of Computer Science as satisfying the thesis requirement. for the degree of Master of Science,Andries van Dam Director. Recommended to the Graduate Council,David H Laidlaw Reader. Robert C Zeleznik Reader,William A S Buxton Reader. Alias Wavefront Inc,Approved by the Graduate Council.
Peder J Estrup,Dean of the Graduate School and Research. Recent approaches to providing users with a more natural method of interacting with com. puter applications have shown that more than one mode of input can be both beneficial. and intuitive as a communication medium between humans and computers Two modali. ties in particular whole hand and speech input represent a natural form of communication. that has been ingrained in our physical and mental makeup since birth In this thesis we. investigate the use of whole hand and speech input in virtual environments in the context. of two applications domains scientific visualization and interior design By examining the. two modalities individually and in combination and through the creation of two applica. tion prototypes Multimodal Scientific Visualization Tool and Room Designer we present. a number of contributions including a set of interface guidelines and interaction techniques. for whole hand and speech input,Acknowledgements, I wish to thank the members of my thesis committee Robert Zeleznik David Laidlaw. Andries van Dam and William Buxton for their support direction and guidance in the. development of the ideas presented in this work In addition I thank IBM for their financial. support for the last two years, I also would like to thank the members of the Brown University Graphics Group for. their endearing support In particular I thank Andy Forsberg Tim Miller Rosemary. Simpson Steve Dollins Tim Rowley Christine Waggoner Mark Oribello Mike Legrand. Brian Perkins Daniel Acevedo Rebecca Sun and Mark Zeldis. Finally I thank my mother father and brother for never letting me get down and. helping me to maintain the energy and drive to finish this work when things got tough. List of Tables viii,List of Figures ix,1 Introduction 1. 1 1 Objective 1,1 2 Contributions 2,1 3 Reader s Guide 3.
2 Whole Hand Input 5,2 1 Previous Work Using Whole Hand Input in VEs 5. 2 2 Whole Hand Input Classification Systems 7,2 2 1 Sturman s Whole Hand Input Taxonomy 7. 2 2 2 MIT AHIG s Gesture Classification System 9, 2 3 Geometrical Topological Hand Data Classification 10. 2 4 Flex and Pinch Input 11, 2 5 Interaction Techniques Using Flex and Pinch Input 12. 3 Speech Input 17,3 1 Types of Speech Input Systems 17.
3 2 Practical Issues with Speech Input 18,3 3 Speech Input Solutions 19. 4 Combining Whole Hand and Speech Input 23,4 1 Multimodal Interaction 23. 4 2 Previous Work 25, 4 3 Advantages of Combining Whole hand and Speech Input into Multimodal. Interfaces 26,5 Hardware and Software Frameworks 28. 5 1 Hardware Configurations 28, 5 1 1 Rear Projected Display Table Configuration 28.
5 1 2 Surround Screen Configuration 30,5 2 Software Architecture 31. 5 2 1 Pinch Glove Finite State Automata 32,5 2 2 SuperGlove Posture Recognizer 33. 5 2 3 CyberGlove Posture and Gesture Recognizer 34. 5 2 4 Left and Right Hand Tracker Data Managers 35. 5 2 5 Speech Token Recognizer and Parser 36,5 2 6 Integration Component 37. 6 Application I Multimodal Scientific Visualization 39. 6 1 Application Functionality and Interaction 39,6 1 1 Navigation 40. 6 1 2 Dataset Manipulation 42,6 1 3 Tool Creation and Manipulation 43.
6 1 4 Recording and Playback 45,6 2 User and Prototype Evaluation 46. 7 Application II Multimodal Room Layout and Interior Design 49. 7 1 Furniture Database Description 50,7 2 Application Functionality and Interaction 52. 7 2 1 Navigation 53, 7 2 2 Furniture and Interior Decoration Creation 55. 7 2 3 Furniture and Interior Decoration Manipulation 56. 7 3 User and Prototype Evaluation 57, 8 Interface Guidelines and Interaction Techniques 60. 8 1 Interface Guidelines 60,8 2 Summary of Interaction Techniques 62.
9 Conclusions and Future Work 63, A Introduction to Hand Posture and Gesture Recogition 65. B Hand Posture and Gesture Recognition Technology 68. B 1 Data Collection for Hand Postures and Gestures 68. B 2 Data Collection Using Trackers and Instrumented Gloves 69. B 2 1 Tracking Devices 69,B 2 2 Instrumented Gloves 70. B 3 Vision Based Technology 78, B 4 Advantages and Disadvantages of Glove and Vision Based Data Collection. Systems 80, C Hand Posture and Gesture Recognition Techniques 83. C 1 Feature Extraction Techniques 83,C 1 1 Simple Feature Extraction and Analysis 85.
C 1 2 Active Shape Models 87,C 1 3 Principal Component Analysis 89. C 1 4 Linear Fingertip Models 91,C 1 5 Spatio Temporal Vector Analysis 92. C 2 Hand Posture and Gesture Classifiers 94,C 2 1 Template Matching 94. C 2 1 1 Classical Template Matching 94,C 2 1 2 Instance Based Learning 96. C 2 1 3 The Linguistic Approach 99,C 2 1 4 Appearance Based Motion Analysis 100.
C 2 2 Statistical Methods 101,C 2 2 1 Hidden Markov Models 101. C 2 3 Miscellaneous Classification Algorithms 105,C 2 3 1 Neural Networks 105. C 2 3 2 Causal Analysis 108,D Flex and Pinch Input Electronics 110. D 1 Flex and Pinch Components 110,D 2 Design and Implementation of Electronics 110. D 3 Electronics Pseudocode 111,Bibliography 112,List of Tables.
C 1 The abbreviations for the feature extraction and classification algorithms. discussed in Appendix C They are referred to in Tables C 2 and C 3 and. Figure C 1 84, C 2 A summary of the feature extraction and classification algorithms found in. Appendix C The table shows information about whether a technique has. been used in a glove or vision based solution the extent of the training. required and how much work has been done using the technique The key. to the abbreviations is found in Table C 1 84, C 3 A correlation between the different feature extraction techniques and the. classification algorithms Each applied entry has either one or two codes. associated with it Each consists of 3 letters a number and then another. letter The first letter states what the posture or gesture set size is the second. letter says whether the set was simple or complex and the third letter says. whether we are dealing with postures or gestures The number shows the. highest reported accuracy number for that particular configuration and the. last letter in parentheses states whether the configuration was done using. a glove or vision based solution The key to the abbreviations is found in. Table C 1 85, D 1 The listed parts that make up the Flex and Pinch electronics unit 110. List of Figures,2 1 The Lifting Palm object selection technique 6. 2 2 The Framing Hands object selection technique 7. 2 3 The Geometrical Topological hand data classification scheme The letters. inside the quadrants are acronyms for a particular component For example. NT stands for a non invasive approach using topological data while IG IT. stands for an invasive approach which uses both geometrical and topological. 2 4 The Flex and Pinch input system The cloth contacts represent the pinch. part of the device collecting discrete topological data while the glove rep. resents the flex part collecting continuous geometrical data Although a. CyberGlove 142 is shown any bend sensing glove can be used 13. 2 5 Two examples of how the cloth contacts can be placed on the hand when. using the head crusher object selection technique 14. 2 6 An example of how the cloth contacts can be placed on the hand using the. lifting palm object selection technique 14, 2 7 A user wearing the Flex and Pinch input device is about to invoke the head.
crusher object selection technique on a round table By placing his middle. and index finger together as shown in the drawing the user can activate the. selection operation and move the table 15, 2 8 A user pointing at and selecting a desk in the virtual environment The user. makes the selection by pressing the thumb to the right side of the middle. finger as shown in the drawing 15, 5 1 The various components that make up the rear projected display table con. figuration 29, 5 2 A Barco Baron rear projected display device 29. 5 3 The various components that make up the surround screen hardware config. uration 30, 5 4 A conceptual model of the TAN VR CUBE display device 31. 5 5 The components that make up our multimodal interface framework Arrow. indicate direction of data flow 32, 5 6 An example finite state machine created within our multimodal interface.
framework The arcs are event transitions which send the interaction tech. nique into a particular state In this case there are three states and a total. of 10 event transitions 33,5 7 An example Hark parameter file 38. 6 1 A user interacting with a dataset for visualizing a flow field around a space. shuttle The user simultaneously manipulates the streamlines with his left. hand and the shuttle with his right hand while viewing the data in stereo 40. 6 2 The grammar file which holds the speech command descriptions used in the. MSVT application Note that words in brackets are optional in issuing a. given voice command 41, 6 3 The three basic components of the two handed navigation technique The. boxes represent the user s hands and the line styles represent possible mo. tions These components can be used in isolation or by combining them so. the viewing region can be scaled rotate and translated in one motion Note. that using one hand at a time also allows for translation 42. 6 4 The rake visualization tool which is made is made up of a number of streamlines 43. 6 5 The user extends his right hand to the display asking for a streamline 44. 6 6 The user is in recording mode as indicated by the red background 45. 6 7 The user watching a previously recorded animation indicated by the green. background 46, 7 1 A simple conceptual model of the first floor of a house used in the Room. Designer application 50, 7 2 An iconic representation of 40 of the primitives present in the furniture. database 51, 7 3 A living room constructed with Room Designer 52.
7 4 The user in a proposed layout for his bedroom 53. 7 5 A simple kitchen created in Room Designer 53, 7 6 The grammar file which holds the speech command descriptions used in the. Room Designer application Note that words in brackets are optional in. issuing a given voice command 54, 7 7 The chair s bounding box is highlighted indicating the virtual sphere has. intersected it A user can pick up the chair or move towards it 57. 7 8 A user deletes a chair by throwing it over his shoulder 58. B 1 The 17 joints in the hand and the associated 23 degrees of freedom from. Sturman 133 71, B 2 The 5DT Data Glove developed by Fifth Dimension Technologies The glove. measures seven DOF from Fifth Dimension Technologies 45 74. B 3 Nissho Electronic s SuperGlove input device worn by the author This glove. has a minimum of 10 bend sensors and a maximum of 16 74. B 4 Fakespace s Pinch Glove input devices worn by the author The gloves have. electrical contact points that allow users to make pinch postures that can. be then mapped to a variety of tasks 75, B 5 The various motions that the hand and fingers can make using its 23 degrees. of freedom from Sturman 133 77, B 6 Virtual Technologies CyberGlove worn by the author which can be equipped.
with 18 or 22 bend sensors 78, C 1 A graph showing posture and gesture set sizes and accuracies for different. feature extraction classification combinations The key to the abbreviations. is found in Table C 1 86, C 2 The user s hand is being tracked with an active shape model Taken from. Heap 57 88, C 3 The dots represent points in the data set while the solid line represents the. axis of greatest variance first principal component The dashed lines rep. resent potential classification divisions 90,C 4 A four state Bakis HMM 103. Introduction, Interaction represents one of the most important components in virtual environment 1 VE.
applications there have been many interface devices techniques and models that have. been researched and analyzed for the purpose of finding usable and robust VE interfaces. One interface style that has shown potential in creating useful and robust interfaces is. multimodal interaction Although multimodal interfaces have existed in computer UI s. since the early 1980 s with Bolt s Put That There system 13 they have only recently. since the early 1990 s begun to be examined and incorporated in virtual environments. and other 3D applications 2, There are many different types of individual modalities that can be combined to form. multimodal interfaces 27 76 144 Two of the most interesting are whole hand and speech. input since these modalities represent a natural form of communication that has been. ingrained in our physical and mental makeup since birth On a person to person level. humans use these modalities in everyday conversation so an interesting question arises as. to the best way to use whole hand and voice input in virtual environments on a human to. computer level,1 1 Objective, The main objective of this thesis is the development of a set of practical guidelines and. interaction techniques for using whole hand and speech input in virtual environment appli. cations We focus on two domains 3D scientific visualization and interior design with the. Virtual environment and virtual reality are used synonymously throughout this document. One could consider Bolt s system a VE application since users are placed in front of a large rear projected. screen and interact using magnetic trackers However Put That There was a 2D application and had. no stereoscopic viewing, hope that the guidelines and techniques reported can extend into other VE applications. In order to achieve this objective it is important to understand not only how whole. hand and voice input can work together but also how they can and have been used in. unimodal virtual environment interfaces An understanding how to improve upon these. individual modalities inherently strengthens them when they are combined multimodally. Therefore this work also analyzes the issues involved with using whole hand and speech. input in isolation resulting in the development of a number of solutions to problems with. these individual modalities,1 2 Contributions, The contributions in this work are presented under four categories which are. 1 Interaction Analysis, A survey into the issues involving whole hand input with specific interest in.
posture and gesture recognition, A classification scheme for the information gathered with whole hand input. A classification of speech input methods with a discussion of the problems and. their solutions for using speech recognition,2 Input Devices. Flex and Pinch input a hybrid whole hand input device. 3 Interaction Techniques, A framework for combining whole hand input and speech into a multimodal. A discussion and implementation of interface techniques integrating both whole. hand and speech input,4 Applications,A fluid flow visualization application. A conceptual modeling furniture layout application. 1 3 Reader s Guide, Since many of the chapters in this thesis contain background information and novel work.
this section presents the reader with a brief description of each chapter s contents and iden. tifies novel sections in parentheses, Chapter 2 Discusses the use of whole hand input in virtual environments and prior. whole hand data taxonomies presents the geometrical topological whole hand input clas. sification scheme Section 2 3 details on Flex and Pinch input Section 2 4 and discusses. some improvements to existing whole hand interaction techniques Sections 2 5. Chapter 3 Discusses the problems and issues associated with speech recognition in virtual. environment applications and presents methods for solving these problems Section 3 3. Chapter 4 Discusses the combination of whole hand input and speech in multimodal. interfaces and identifies a number of advantages Section 4 3 for using multimodal inter. action in virtual environment applications, Chapter 5 Presents the hardware configurations and software framework Section 5 2. used in implementing the two applications described in Chapter s six and seven. Chapter 6 Discusses the motivation features interaction techniques Section 6 1 and. qualitative evaluation of a scientific visualization application for viewing flow around a. Chapter 7 Discusses the motivation features interaction techniques Section 7 2 and. qualitative evaluation of a room layout interior design application which allows users to. populate naive environments, Chapter 8 Presents a set of guidelines for using whole hand input and speech in virtual. environments Section 8 1 and a summary of the novel and improved interaction techniques. Section 8 2, Chapter 9 Presents conclusions and areas for future work.

Related Books

HISTORIA GERAL da AFRICA - UNESCO

HISTORIA GERAL da AFRICA UNESCO

Investigation on Transient Stability of Six-Phase ...

Investigation on Transient Stability of Six Phase

International Conference on Innovations in Electrical & Electronics Engineering ... (EPRI), is to create the ... (FACTS), high-voltage dc (HVDC) ...

A LITERATURE REVIEW ON MATERNAL MIGRATION: CASES OF CHINA ...

A LITERATURE REVIEW ON MATERNAL MIGRATION CASES OF CHINA

A LITERATURE REVIEW ON MATERNAL MIGRATION: CASES OF CHINA, SRI LANKA, GUATEMALA, PHILIPPINES, AND POLAND ... either as an avenue to the enhancement of the standard of living or as the result of the imposition ... (Sundari, 2007), migration is often regarded as male-centered, resulting in neglect and bias and for female migration to be ...

Implementing Quality and Safety at the Unit Level in an ...

Implementing Quality and Safety at the Unit Level in an

male and one panelist representing Americans with disabilities or English as a second language (ESL) students. ... team contributions in the enhancement of care was ignited. Informatics . Very early in the process, students involved in these projects ... and expands access to safe, quality APRN care. The paper defines APRN practice, describes ...

Honors Physics Semester 2 Final Exam Review Answers

Honors Physics Semester 2 Final Exam Review Answers

Honors Physics Semester 2 Final Exam Review Answers Page 4 Pictured below is a new roller coaster. A physics student riding and the coaster car have a combined mass of 200 kg. The car is at rest at point A and there is no braking at point D. Assume no friction between the coaster car and the track. 41. Rank the total energy from least to greatest.

Berkeley Main Post Office Public Meeting

Berkeley Main Post Office Public Meeting

Public Meeting Written Comment Period ... 25 50 75 100 125 2006 2010 2016 2020 103 83 82 81 0 25 50 75 100 125 2006 2010 2016 2020. First-Class Mail Standard Mail . Financial Climate of Postal Service . Mail Volume Shifting to Less Profitable Mix . Volume in Billions of Pieces (Source: Boston Consulting Group) 6 Net Losses of over $25 billion past five years, despite removing nearly $14 ...

Undergraduate in of in Office Location: UHB 3038 ...

Undergraduate in of in Office Location UHB 3038

University of Illinois Springfield ... Through careful self-assessment and goal-setting, Liberal Studies ... such as international studies or human resource ...

Blaming the victim: assessment, examinations, and the ...

Blaming the victim assessment examinations and the

always exerted a controlling in?uence over the school curriculum ... including empirical studies of assessment ... with neo-liberal arguments about human resource ...

Machine Learning in Computer Vision - Higher Intellect

Machine Learning in Computer Vision Higher Intellect

Machine Learning in Computer Vision by N. SEBE University of Amsterdam, The Netherlands IRACOHEN ASHUTOSH GARG and THOMAS S. HUANG University of Illinois at Urbana-Champai gn, HP Research Labs, U.S.A. Google Inc., U.S.A . Urbana, IL, U.S.A. AC.I.P. Cata logue recor d for this book is available from the Library of Congress. Published by Springer, P.O. Box 17, 3300 AA Dor drecht, The Netherlands ...

A multimodal approach to distinguish MCI-C from MCI-NC ...

A multimodal approach to distinguish MCI C from MCI NC

Orientador Interno: Professor Dr. Nuno Matela, Instituto de Biof sica e Engenharia Biom edica ... Nos exerc cios de CAD, em particular quando se usam t ecnicas de