Aug 19, 2010 - protein engineering techniques such as site-directed muta- genesis (1) .... Binary files (such as images, pdf, zip or other media files) can be ..... having a sequence, structure or other data associated it with it. The screenshot pres
Dec 6, 2002 - woke, he said to himself, 'Am I Zhuang Zi dreaming I was a butterfly or am I a butterfly dreaming of ...... been returned to the basement and as far as I'm aware, they have not been played since. The new ...... framing of the piece was
counts may involve ecologists making ten-minute observations at dawn, noon and dusk over a period of five days at a single site. ..... Oscillating single syllables in time domain. Detection of temporal oscillations within a characteristic frequency b
It is an interesting era to be working in the VLSI testing ... Changes in IC products, processing technology and IC market dynamics suggest that conventionally used test methods (e.g., at- speed functional testing) must significantly change in the fu
This paper presents two experimental comparisons of data ow and mutation testing. These two techniques are widely considered to be e ective for unit-level software testing, but can only be analytically compared to a limited extent. We compare the tec
established methodologies and practices cannot always be applied successfully in the changing international research .... With respect to the nature of âgoodâ qualitative research many authors argue that it should be easily replicable. ..... data
data. Leading ORB vendors have de ned and imple- mented IDL-Java mapping allowing the invocation of. CORBA objects from Java applications. Examples of these products are Joe ... Users are incremen- tally and dynamically educated about the available .
Sep 10, 2015 - practice (Glatard et al., 2015; Poline et al., 2012). At the same time, cross-site data sharing .... with LAMP stack, while CBRAIN is a Ruby on Rails application. Installa- tion and customization of either .... growing amounts of resti
Jun 1, 2011 - In 2007, the National Human Genome Research Institute (NHGRI) established the Electronic MEdical Records and. GEnomics ... This paper discusses the challenges of sharing genomic data linked to health information in the electronic .... c
of 98 scholarly papers on the topic and b) a survey among 603 secondary data users who are analyzing data from ... Wolf et al., 2006 (also ProQuest); Feldman et al., 2012; Harding et al., 2011; Jiang et al.,. 2013 .... Q1 We are considering giving SO
municipal departments. Property valuation rolls at municipalities. Property description(as per deeds registry) together with a postal address. Property valuation ... such as debt collectors, media companies, banks, insurance companies, ... data colle
forced-choice experiments, within-pair order effects. 1 Introduction. Paired comparison studies are utilized in various fields of study, including marketing, trans- portation, environmental and health economics, psychology and sensometrics. This is b
Sep 1, 2014 - The explosion of the data both in the biomedical research and in the healthcare systems demands urgent solutions. In particular, the research in omics sciences is moving from a hypothesis-driven to a data-driven approach. Healthcare is
Jan 15, 2013 - Section 2 describes the main requirements of the intended information system from the point of view of the opinion posts, their integration into the corporate data warehouse, the analysis of the combined data and the quality of the obt
Sep 23, 2002 - Researchers submitting to the PROMISE conference on predictive models in software engineering are ... previous results, but also in providing an insightful understanding of the fundamentals. âCorrespondence to: .... Text mining data
Jan 5, 2010 - Analysing and representing narrative data. At this stage I was dissatisfied with the writing up process and was having difficulty finding one phrase or title that summarised the main theme of the thesis. I experimented with a number of
Analysing phenomenological data and writing up. Above all else, phenomenologists ... A key process involved in analysis is the act of 'dwelling' with the minutiae of data. âWhen we stop and linger with ... stage of analysis we empathically join wit
Research and Practice in Information Technology, Vol. 28. A. Cockburn ... table display, and a connected participant 3 working behind a monitor. The virtual table could seat all participants on separate sides, or have the participant. 3 seated on the
the research methodology; Section 4 gives an example of the analysis using one of our data sets; Section 5 presents the results of the meta-analysis; Section 6 presents an investi- gation of nonlinear transformation of the responses and its influence
widely shared premise that existing data about institutional change provide, at best, a grossly insufficient ..... in many important industries and, when it takes place among firms with market power (which is nearly always the case), ... One specific
Mar 29, 2013 - interdisciplinary research, student training, international exchange of ideas, and transfer of science into ... (1) instigate further exploration of this data to advance ba- sic understanding of river systems and its ... upstream end o
analysis. Moving past DNA (deoxyribonucleic acid) sequences, researchers are interested in the corre- sponding protein sequences, their structure, and func- tion. Beyond sequences .... answers from the various data sources to produce an answer to the
This paper describes experiments in applying data mining techniques to historical data collected by network ... difficult to manage the complex interaction between different components such as hosts, devices ... The Spectrum Enterprise Management Pla
Sharing and Analysing Presence Experiments Data Doron Friedman1, Andrea Brogni1, Angus Antley1, Christoph Guger2 and Mel Slater1 1 Department of Compute Science, University College London, UK 2 g.tec - Guger Technologies OEG, Austria. Abstract Presence research heavily relies on empirical experiments involving subjects in mediated environments. Such experiments can be extremely resource intensive and produce very large amounts of data. As the presence community matures, we would like to suggest that data collected in experiments will be publicly available to the community. This will allow the verification of experimental results, comparing results of experiments carried out in different laboratories, and evaluating new data-analysis methods. In this paper we present the complete data set from a large-scale experiment that we have carried out in highly-immersive virtual reality. We describe the data we have gathered and give examples of the types of analysis that can be made based on that data. Keywords--- methodology, presence, virtual reality, physiology, GSR, ECG.
1. Introduction Engineering practice places emphasis on reusable components and standardization. In Presence research technical standardization and reusable components, such as reusable virtual environments (VE) would be useful, but moreover, we can strive towards standardization in data collection and data analysis. This will serve the following goals: x Researchers can test new methods of analysis without carrying out time-consuming experiments. x Researchers who have experiments with huge amounts of data can rely on the community to assist in the analysis. x This will ensure high quality of experiments and publications. x This will promote progress in presence methodology, by allowing a comparison of methodologies and research techniques. x It would make it possible to compare results among different experiments, and even among experiments carried out in different labs. In this paper we present the complete data for an experiment we have carried out in a Cave-like system1. The experiment was large scale in that it included several types of measurements, including both quantitative data such as physiological measurements and qualitative data such as
semi-structured interviews. The hypothesis and the results for this specific experiment are not of interest here. We describe the data itself and point to the analysis techniques we have used to analyze it. We suggest other techniques to analyze the same data, and some other types of data that may be used in the future. The IPQ group have already published data from presence experiments and encouraged other researchers to use it2. However, this only includes data from the IPQ questionnaire; we encourage publishing and sharing all types of data. Analyzing data is, of course, not unique to presence research. Research on evaluating VR usability in general may be relevant here (e.g., ). The unique characteristics of our research, as well as the research of many others in the presence community, is that we have different types of data generated by technical devices and computers, which are mostly detailed and accurate, and ultimately, their integration can allow us to “reconstruct” the subjective experience of the subject. One area which is similar in many aspects, and in which such reconstruction of experience is considered critical, is post-mission debriefing of air-force pilots (e.g., see ). The data for the experiment, as described in this paper, can be downloaded from: http://www.presencedata.info/. The data is organized in online tables. In the paper below we refer to the data by mentioning the table number; these can be accessed from the main URL above.
2. The Experiment As mentioned earlier, in this paper we are not concerned with the specific hypothesis or with the results of the experiment; these are described elsewhere [12, 14, 25]. Rather, we describe the types of data collected and the ways to analyze them. In this section we provide an overview of the experiment goals and procedure. The overarching goal of the experiment was to investigate presence as a multi-level construct ranging from lower-level involuntary responses to higher-level subjective responses. Specifically, this experiment was designed to find physiological correlates to breaks in presence (BIPs) [3, 23, 24, 26]. However, note that the techniques mentioned in this paper should be appropriate for most presence experiments. Upon arrival, participants were given an instruction sheet describing the experimental procedure and the possible risks associated with using virtual reality equipment (including simulator sickness). They were asked
CAVE¥ is a trademark of the University of Illinois at Chicago. In this paper we use the term ‘Cave’ to describe the generic technology as described in , rather than to the specific commercial product.
PRESENCE 2005 to fill out a consent form and a pre-questionnaire covering their age, gender, occupation, and previous experience with VEs and computer games. They were then led though to the Cave, where they were shown how to connect the electrocardiogram (ECG) and respiration sensors. Galvanic skin response (GSR) sensors were attached to their non-dominant hand, and they were asked to stand still in the Cave for a baseline reading. During this time, no images were displayed on the Cave walls. Next, participants were asked to complete a brief exercise in a virtual “training” room designed to make them comfortable moving around the Cave. Once they felt comfortable, they were told that in a few moments they would find themselves in a bar, where they were asked to spend a few minutes until we told them it was time to come out. It was explained that they were free to explore the bar as they wished, and that afterwards we would be asking them questions about the experience. They remained in the virtual bar for the duration of two songs, approximately five minutes. Note that the virtual space of the bar was not much larger than the physical space of the Cave; this means subjects moved around the bar by walking rather than navigating with a wand. The bar contained five virtual characters: one barman, one couple standing near the bar on the right, and another couple seated on the left of the room. When approached by the participant, the characters would utter phrases suggesting that a celebrity was about to arrive. At four points during the experience, the walls of the Cave were blanked out, leaving participants in a completely white room for approximately 2 seconds. Two experimental minders observed them throughout, noting their bodily and verbal responses to the whiteouts. Participants’ autonomic responses were also monitored throughout. Figure 1 shows a participant in the bar environment, wearing the physiological monitoring equipment.
Figure 1: Participant wearing the bio-sensors and VR goggles in the Cave. The experiment included two conditions. The main condition included 20 subjects and the goal was to try to detect, for these subjects, whether there is a physiological “signature” to the BIPs. A second condition included 10
other subjects. They went through the same procedure as described above, but in addition they were given an explanation about BIPs, or “transitions to real”. They were trained to click a wireless mouse whenever they experienced a transition from the virtual world to the physical reality of the laboratory. During the experiment phase, they were asked to click the mouse whenever they had such a transition. The goal was to find out if it was possible to detect a physiological signature to these selfreported BIPs. Immediately after the experience, and before taking off the equipment or leaving the Cave, participants were asked to answer two questions concerning their immediate impressions regarding their overall sense of “being in” and “responding to” the bar. Next, they were shown the video of themselves in the bar, and were asked to comment on anything that they remembered while watching the video. A semi-structured interview was conducted afterwards. The experiments were carried out in a four-sided Cavelike system , which is driven by an Onyx IR2 with 4 graphics pipes. Subjects were wearing Intersense IS900 wireless trackers. The application was written on top of Dive [6, 27]. Physiological signals were measured using ProComp Infiniti by Thought Technology Ltd.
3. The Data Analysis methods are typically classified into quantitative and qualitative methods. We do not undermine this distinction, but in this paper we find it useful to make another distinction: between data that is temporal and data that is not.
3.1. Temporal Data It has been argued that rather than being a stable constant throughout the mediated experience, presence may vary over time [1, 16, 20]. Generally, we would like to be able to measure how presence varies over the duration of the experience, and how it is affected by specific events in the environment. Specifically, we encourage studying presence by looking at physiological data. One of the first studies to show that presence can be studied as an objective, measurable response, based on GSR and heart rate, was carried out by Meehan et al. . Ideally, all data could be placed on the same timeline, and visualized together. In this section we present these types of data independently, and in later sections we discuss possible ways to cross-analyze them. Most of the temporal data are generated digitally; the main challenge is synchronization. Accurate synchronization is critical for event-related responses, such as discussed in section 4.2. In our lab, we use the Virtual Reality Peripheral Network (VRPN)3 to synchronize among the data and the VR system. VRPN is an open software platform that is designed to implement a networktransparent interface between application programs and the 3
PRESENCE 2005 set of physical devices (trackers, etc.) used in a VR system. VRPN was recently extended to support the ProComp physiological recording device. Using VRPN, all data generated during an experiment can be synchronized, sent over a network, and stored with uniform timestamps for later analysis. 3.1.1. VE Events: In VR, and in fact in any type of digital media experience, it is possible to keep accurate logging of most meaningful events that take place during a session. First, events and actions carried out by the system can be easily logged by the application. In our experiment we recorded all the instances in which the virtual characters spoke. The data is included in Online Tables 1A and 1B. Second, events carried out by the participant typically involve some type of interaction device; such events are easily tracked as well. In our experiment, we have allowed, in one condition, for subjects to indicate breaks-inpresence, using a wireless mouse device. The data is included in Online Table 1C. Typically, VEs would allow interactions of subjects with the VE; such events would similarly be tracked and logged. We did not record when participants speak, but this could be done in principle. The analysis of this type of data is typically useful for detecting event-related responses. For example, one can look at the physiological state of the participant whenever something happened in the VE; examples are given in section 4.2. 3.1.2. Tracker data: In VR the participants are, typically, head tracked. This provides extremely useful information about their position and head direction at any moment. While theoretically they can be looking sideways, we expect this gives us a good approximation of what they were looking at, without the need to perform eye tracking, which is difficult in a Cave environment. The tracker data from our experiment is included in Online Table 2. Examples of this analysis are spatial analysis (see section 4.3) and event-related analysis (see section 4.2). We did not use head-tracker data to reconstruct what the subjects were looking at; this should be possible since the trackers include orientation information as well as position. 3.1.3. Galvanic Skin Response: GSR, also sometimes called galvanic skin conductivity or Electro Dermal Activity (EDA), is measured by passing a small current through a pair of electrodes placed on the surface of the skin and measuring the conductivity level. In our experiment GSR was sampled at 32 Hz, and the signal was obtained from electrodes on two fingers. The GSR data for the two experimental conditions appears in appendices 3A and 3D. More details about GSR, and about analyzing GSR data from this experiment, can be found in Slater et al. . Specifically, they show that the GSR parameters predict the occurrence of breaks in presence, using a method based on continuous wavelet transforms of the GSR signal.
3.1.4. ECG: Several parameters can be extracted from ECG recordings. In addition to the obvious one – heart rate – the heart-rate variability (HRV) can be used to describe the physiological behavior of the participant, and an eventrelated heart-rate response may be useful to study the reaction of the subject to an event (such as a BIPs). The ECG data for the two experimental conditions is provided in appendices 3B and 3E. The sampling rate is 256Hz. Slater et al.  and Guger et al.  provide an analysis of the ECG, including a comparison of the training and experimental phases, comparison of social phobic and non-social phobic participants, and event-related ECG. 3.1.5. Respiration: The respiration signal measures the inhalation and exhalation phases of the human subject. The signal can be used to extract the deepness and frequency of the respiration. The first step is to low-pass filter the signal with 10 Hz to remove noise components and movement artifacts. Then each zero crossing of the bipolar respiration signal is detected in order to calculate the frequency. Eventrelated respiration changes around a BIP can be investigated. For example, it is possible to detect a change in deepness and frequency after the BIP. It is very common that the subjects hold their breath for a few seconds when the BIP occurs. The respiration also modulates the ECG signal with a frequency of about 0.1-0.2 Hz. This modulation effect must be considered when the ECG is analyzed; details can be found in Florian et al. . The respiration data for the two experimental conditions appear in appendices 3C and 3F. 3.1.6. Video: The whole experiment session was videotaped for all subjects. In our experiment we used a Cave system where the projection takes place on three walls and on the floor. The camera was placed outside the Cave so that it captures the whole area of the Cave. This is useful to observe the subject’s motion throughout the physical space of the Cave, and also allows analyzing their main body gestures and postures. However, the subject is typically shown from the back. Generally, it would be difficult to pick up the subject’s facial expressions, given the relative darkness in the Cave and the fact that subjects wear VR goggles. We still recommend placing another camera that picks up the subject from the front; e.g., in our Cave setting, we could eventually place one on the top of the front Cave screen. The video can be used for testing hypotheses, for providing the experience to researchers who were not present in the experiment, and for later analysis of body language. A sample of the videos can be found in Online Table 4 and a copy will be sent upon request. 3.1.7. Video interview: Following the experiment, the subject watched the video together with the experimenter and reflected on his or her experience. We have used this video interview to gain some insights for later exploration during the post-experiment interview. Ideally, this interview by itself should be recorded and provided with the data, because it provides a potentially insightful glance
PRESENCE 2005 into the subject’s experience when it is still fresh, and in a way that allows the subjective impressions to be temporally aligned with the experience.
interaction sessions. Such tools may allow one to view the interaction from various angles, and to focus on specific events within a session .
3.1.8. Additional measurements: In the future we hope to explore additional types of temporal data. Some experiments involve conversation, either among multiple subjects in a multi-user experiment or between a subject and a confederate. Recording such conversation and synchronizing it with the other types of data can be extremely useful. This is a specific rich type of VE events, as discussed in Section 3.1.1. Other types of physiological data can also be used. Our system now includes electroencephalograph (EEG) measurement as well; this was used for a brain-computer interface , but may also be used for post-experiment analysis. Similarly, it should be possible to analyze muscle activation in the form of electromyogram (EMG) recording. Such measurements are extremely useful for measuring emotional states4, and may be especially useful in VR where the subject’s face is obstructed by the VR goggles, which undermines video-based facial expression analysis. It is also possible to track body parts in addition to the head. Most VR systems include a wand device that is tracked. The VE of the bar room was such that no navigation using wand was required; walking in the Cave was enough. Thus, hand-tracking data for this experiment is not available. For some experiments it may be useful to include the wand, even as a simple tracking device; it may be possible to partially analyze hand and arm gestures. Naturally, full-body tracking is highly useful for experiments that may involve body language and nonverbal communication. If such tracking devices are not available, it is still possible to utilize experts in body language who can observe the subjects and interpret their behavior; this can be done after the experiment by watching the video5. Freeman et al. studied postural shifts in response to motion stimuli , as an indication of presence. In addition to documenting the experiment sessions by video, it would be useful to be able to record the virtual environment. Such recording of interactive environments, although not a new idea, is still not straightforward and is not provided by any of the standard VR toolkits. There are a few systems that allow users of VR/VE systems to review sessions [13, 15, 17], and Steed et al.  actually used such a system in their experiments. They describe a system that records the full Dive session and allows the experimenter to play it back within Dive and experience it as a first person view. It is also possible to use intelligent tools that create movie summaries from
3.2. Non-Temporal Data
Hugo Critchley, UCL Institute of Neurology, private communication. 5 Note that we ensure that the subjects are completely separated from the surrounding lab by covering the Cave with curtains. This means the experimenters can only observe the subjects during the experiment by watching the video feed from the camera. Thus, there is in practice no difference between analyzing the experiment (video) during the experiment or afterwards.
In this section we discuss data that is collected after the experiment, and thus cannot be temporally aligned with data collected during the experiment. 3.2.1. Questionnaires: It has been pointed out several times that questionnaires are problematic in the context of measuring presence: for example, they are unstable, in the sense of being very sensitive to prior experience , they may not be able to distinguish reality from virtual reality , and they can shed no light on whether ‘presence’ actually exists as a uniquely identifiable brain activity during the course of the experience to which it is meant to relate . Questionnaires may be made more useful and reliable if their results are integrated with qualitative results and with physiological data, such as suggested in this paper. Even if the community does not converge on one questionnaire, experimenters could let the subjects fill in more than one questionnaire, thus allowing crosscommunity comparisons. In addition to presence questionnaires, we suggest administrating psychological tests, such as personality tests. Again, these tests could be controversial independently, but could have valuable contribution when crossed with other data. As an example, Slater et al.  found a correlation between a test for social phobia and ECG. The results for the psychological test that elicits the degree of social phobia  are given in Online Table 5C. In the bar experiment only the subjects in the second condition filled in presence questionnaires; the questionnaire appears in Online Table 5B and the results are available in Online Table 5C. We are now refining a methodology of evaluating and measuring presence based on a combination of questionnaires, interviews, and physiological responses; thus, in the future, we do plan to include presence questionnaire data with our experiments. Each participant also completed a questionnaire prior to their immersion that gathered basic demographic information and other background information regarding their use of computer games. The questionnaire is included in Online Table 5A, and the results in Online Table 5C. 3.2.2 Immediate question: Immediately after the experience, and before taking off the equipment or leaving the Cave, participants were asked to answer two questions concerning their immediate impressions regarding their overall sense of “being in” and “responding to” the bar. The purpose of these two questions was to capture participants' immediate subjective response to the experience in a way that was as far as possible unclouded by post-hoc rationalizations. Afterwards, they were able to expand on their answers in the semi-structured interviews. The responses to the immediate questions are given in Online Table 6C.
Figure 2: Presence graph illustrating BIPs (P8 female) . While the diagrams provide some temporal information, they cannot be aligned precisely with the temporal data, and thus are not considered here to be temporal. Ideally, they could serve as a link between the interview and the temporal data. For example, while drawing the diagram, the subject can point to certain extreme points in their presence function and describe how they relate to their interview. The presence graphs were abstracted and classified into four types ; the data is provided in Online Table 6C. 3.2.4 Post-experiment interview: this is often very useful in providing hypotheses for future research. Such interviews typically contain a lot of fascinating insights, which often gets lost because, due to their subjective nature, they are difficult to analyze in a rigorous manner. Garau  and Thomsen  used Grounded Theory extensively for analyzing interviews in the context of presence research. By including the interview transcriptions with the data we hope other researchers can get an insight into the subject’s experience, and perhaps suggest methods of analyzing this data in a systematic way. In our experiment each interview was conducted using a semi-structured interview agenda, to ensure that it did not stray from the research questions in which we were interested. The interviews were audio taped and then transcribed verbatim. In the future, it may be useful to videotape the interviews, for post analysis of the interview itself. Garau et al.  discuss the interview techniques and the results obtained for this experiment. The transcriptions for the two experimental conditions are included in appendices 6A and 6B.
4. Compound Analysis In the previous sections we described the individual data types and the analysis we have carried out with them. In this section we describe analysis of two or more elements together. Again, the intention here is to explain
what types of analysis are possible, rather than to focus on specific results from this experiment.
4.1 Event-Related Analysis It is of great interest to see if we can detect measurable responses to events in the experience. Our experiment was specifically designed to find out if we can detect a physiological “signature” to BIPs; such evidence was found using a wavelet transform of GSR parameters as reported in Slater et al. . Also, using the same analysis technique, they have found a significant physiological response to events in which the virtual characters spoke to the subject. One possibility that we are examining is whether the stiffening stabilizing reaction that subjects have to a sudden change in their visual field is detectable as a loss in height that can be seen in the head tracker data. In the future we could combine this with EMG data from the Soleus muscle in the lower leg to detect when a subject is experiencing a BIP. An example of one subject’s height following a BIP appears in Figure 3. The subject pulls down nearly two centimetres after the BIP. Due to high variance in peoples’ standing height our results are so far inconclusive. However, given that many emotions, such as stress, are manifested as muscular tension in the body, looking at the results of this muscular activity whether through EMG or postural change is a promising method to analyse response to virtual reality. subject's height during a break in presence subject height in meters
3.2.3 Presence graph: During the interview, participants were asked to draw a graph describing the extent to which they felt they were in the bar versus being in the laboratory throughout the experience. A sample graph is shown in Figure 2.
1.6 1.58 1.56
Height of subject
1.52 1.5 1 time in seconds
Figure 3: A graph of one subject’s head height, as measured by the head tracker, following a BIP event.
4.2 Spatial Analysis A spatial representation of time-variant signals is a very useful tool for the experimenter. A quick glance may allow detection of areas of the VE where the signal has extreme values, and this may provide clues for further analysis. Specifically, an interesting approach in the data analysis is linking the physiological values with the position of the subject in the virtual space at the same time. The resulting graph shows how the signal spatially changes over the VE, and it can be useful to detect whether there is
PRESENCE 2005 a difference in the way different areas affect the subjects’ physiology. While such plots may not qualify as conclusive evidence by themselves, they could be a useful starting point for further analysis of physiological responses and proximity to virtual characters. Figure 4 illustrates this point for a few subjects in the bar experiment: Figures 4a and 4b show data for subjects who had stronger GSR values next to the barman, whereas figures 4c and 4d show
subjects with the opposite trend. Of course, many subjects did not show a clear pattern at all This technique is typically more useful when the VE is large, and the exploration of the VE is of interest in itself. In the case of the bar experiment, the room was spatially limited by the Cave’s walls, and movement was restricted. In this case we provide an example of the response to the virtual characters, but it is also possible, of course, to study the response in the vicinity of virtual objects.
Figure 4: Plots of GSR levels in space from four different subjects. The positions of the five virtual characters are given by letters: the barman is denoted by the letter B and the other characters by the letter A. GSR values are denoted by more red colors, and lowest GSR values are towards blue. Interestingly, the qualitative analysis of the interviews seemed to reveal that the subjects responded differently to different areas in the virtual space, as related to the spatial organization and to the virtual characters . It could be interesting to compare this qualitative evidence with the objective physiological measure. Our tracker data definitely seems to indicate that almost all subjects spent more time on the side of the barman than in the left side of the bar.
4.4 Merging Temporal and Non-Temporal Data Subjective descriptions provided by subjects can be insightful; examples from the experiment discussed here appear in . However, how do we cross them with temporal data? We would like to have an equivalent of the interview, which is obtained during the experiment.
PRESENCE 2005 One such option may be to ask the subject to verbalize their subjective experience out loud, during the experience. When teaching drama this is often used as an exercise. At first the students report that it feels unnatural for them to speak out loud, but they quickly get used to it. Is it possible to train subjects to verbalize their thoughts and feelings, during an experience, to an extent that it does not interfere with their experience? We do not know, but we hope to explore this option. Another option is possible if the VE scenario includes a well-defined narrative. If so, subjects can be encouraged, in the interview, to describe their feelings as related to certain events. For example, in a modified bar experiment, subjects can be encouraged to describe their feelings when a character tells them something intimate. Even though a few minutes pass from the time of the real experience to the time of the reconstruction, the information gathered in this way may be useful. Such recollection of the experience may be done during the video interview, as mentioned in Section 3.1.7, or with a replay of the VE events as suggested in Section 3.1.8.
4.5 Inter-Experiment Comparisons Ideally, it should be possible to compare experiments carried out by different researchers in different labs, even if only part of the data is available. This is especially true for qualitative data such as physiological data and questionnaires, which could be assumed to be universal. Obviously, there would be many differences in the settings and the contexts among different labs, and it would not be simple to compare the data. For example, if one group finds a much stronger physiological response to a virtual character than another group, we would need to carry out further work to understand why this happens: is this because of the technical setting of the lab, the software used, the specific task and scenario in the experiments, or something different. Still, we believe such comparisons would be insightful, especially when there are large differences among research groups. We hope to compare the results of this experiment with others. In our lab there have been experiments carried out with a similar experimental procedure but with a different VE. More generally, we can compare the physiological responses in this experiment with the physiological responses obtained in other experiments using the same Cave setting.
5. Discussion In this paper we detail the types of data that we collected during one experiment, and the techniques we used to analyze this data. There is still a long way to set standards for data sharing and analysis in the presence community. We recognize that presence is a complex, multidimensional concept, which needs to be studied with multiple techniques in multiple levels. Thus, we expect that if presence research matures we will be faced with ever
growing1 amounts of data, of different types, which will need to be analyzed. In order for such data sharing to become widespread, there is a need for standards in data representation, and a standard set of tools and utilities that will allow converting the data into the commonly used tools. In this paper we do not yet suggest such standard. We believe it is too early to suggest complex mechanisms (e.g., using XML to annotate the data), but we hope that the next step would be for researchers to define data formats and provide generic utilities that import, export, and analyze data using this formats. Finally, in order to avoid abuse of data, we would need to suggest copyright mechanisms, probably in the lines of GPL6.
6. Conclusions We encourage other researchers to use their methods in analyzing this data. There are large parts of our data that have not yet been analyzed, or only partially analyzed, and we welcome other researchers to apply other techniques to the data that we have already analyzed. We encourage other researchers to publish their data, in addition to their results, as we have done in this paper. This will allow the community to analyze and compare experiments as a shared effort, assigning credit where due, of course. Once a corpus of data is available and arranged in a systematic way, we can strive towards additional analysis methods. In particular we encourage researchers to investigate analysis of physiological data. The integration of the different data types presents an interesting challenge. We hope to address it in the future, using visualization and possibly data mining. We feel this would allow presence research to be established as a genuine scientific discipline by such eventual data publication and sharing7. Finally, we encourage researchers to use this methodology, which relies on large amounts of synchronized recorded data in a mediated experience, beyond presence research; we expect our setting to be useful for researchers in many areas of psychology.
Acknowledgements This work has been supported by the European Union FET project PRESENCIA, IST-2001-37927. We would like to thank Anthony Steed for his comments on this paper. .
For example, as part of the Visible Human Project, the U.S. National Library of Medicine has recently made digitized datasets of male and female human cadavers available for research and education.
PRESENCE 2005 References  F. Biocca, J. Burgoon, C. Harms, M. Stoner. Criteria and scope conditions for a theory and measure of social presence. In Fourth Annual Int’l Workshop: Presence 2001, Philadelphia (PA), USA, May 2001.  D.A. Bowman, D.B. Johnson, L.F. Hodges. Testbed evaluation of virtual environment interaction techniques, Proc. ACM symp. on Virtual reality software and technology, pp 26—33, London, United Kingdom, 1999.  A. Brogni, M. Slater, A. Steed. More breaks less presence. In The 6th Annual International Workshop on Presence. Aalborg, Denmark, 2003.  C. Cruz-Neira, D. J. Sandin, T. DeFanti. Surround-screen projection-based virtual reality: The design and implementation of the CAVE, in Proceedings of Computer Graphics (SIGGRAPH) Proceedings, Annual Conference Series, 135-142, 1993.  G. Florian, A. Stancak, G. Pfurtscheller. Cardiac response induced by voluntary self-paced finger movement. Int’l J. Psychophysiology, 28(3):273-83, 1998.  E. Frecon, G. Smith, A. Steed, M. Stenius, O. Stahl. An overview of the COVEN platform. Presence-Teleoperators and Virtual Environments, 10:109-127, 2001.  J. Freeman, S.E Avons, D. Pearson, W.A. Ijsselsteijn. Effects of sensory information and prior experience on direct subjective ratings of presence. Presence-Teleoperators and Virtual Environments, 8:1-13, 1999.  J. Freeman, S.E. Avons, R. Meddis, D.E. Pearson, W. IJsselsteijn. Using behavioural realism to estimate presence: A study of the utility of postural responses to motion stimuli. Presence: Teleoperators and Virtual Environments, 9(2): 149– 164, 2000.  D. Friedman, Y. Feldman, A. Shamir, T. Dagan. Automated creation of movie summaries in interactive virtual environments. In Proc. IEEE Virtual Reality 2004, pp191--198, Chicago, IL, 2004.  D. Friedman, R. Leeb, A. Antley, M. Garau, C. Guger, C. Keinrath, A. Steed, G. Pfurtscheller, and M. Slater. Navigating virtual reality by thought: First steps. Proc. 7th Annual Int’l Workshop on Presence, PRESENCE 2004, pages 160—167, Valencia, Spain, October 2004.  M. Garau. The Impact of Avatar Fidelity on Social Interaction in Virtual Environments. PhD thesis, University College London, 2003.  M. Garau, H. Ritter-Widenfeld, A. Antley, D. Friedman, A. Brogni, and M. Slater. Temporal and spatial variations in presence: A qualitative analysis. Proc. 7th Annual Int’l Workshop on Presence PRESENCE 2004, pages 232—239, Valencia, Spain, October 2004.  C. Greenhalgh, M. Flintham, J. Purbrick, S. Benford. Applications of Temporal Links: Recording and Replaying Virtual Environments. In: Proc. of IEEE VR 2002 (Virtual Reality), pp 18, November, 2001.  C. Guger., G. Edlinger, R. Leeb, G. Pfurtscheller, A. Antley, M. Garau, A. Brogni, D. Friedman, M. Slater. Heart-Rate Variability and Event-Related ECG in Virtual Environments. In Presence 2004: The 7th Annual International Workshop on Presence, pp. 240-245, 2004.  J.C. Hart. On recording virtual environments, Proc. IEEE Visualization '93 Symposium on Research Frontiers in Virtual Reality, pp. 80-83, Nov. 1993.  W. Ijsselsteijn, J. Freeman, H. de Ridder. Presence: Where are we? Cyberpsychology and Behavior, 4(2): 179– 182, 2001.  T. Imai, A. Johnson, J. Leigh, D. Pape, T. DeFanti. Supporting transoceanic collaborations in virtual environment, In the Proc. of the 5th Asia-Pacific Conf. on Comm. / 4th
Optoelectronics and Comm. Conf. - APCC/OECC '99, pp. 10591062, Beijing, China, October 19-21, 1999,.  R. Lipshitz, M. Popper, N. Ron. Post-flight reviews in the Israeli-Defense-Force air force as an organizational learning mechanism. Presented at the 3rd Int’l Conf. on Organizational Learning, Lancaster University, June, 1999.  M. Meehan, B. Insko, M. Whitton, F.P. Brooks. Physiological measures of presence in stressful virtual environments. Acm Transactions on Graphics, 21:645-652. 2002.  T.B. Sheridan. Musings on telepresence and virtual presence. Presence: Teleoperators and Virtual Environments, 1:120–125, 1992.  M. Slater. Measuring presence: A response to the Witmer and Singer presence questionnaire. Presence-Teleoperators and Virtual Environments, 8:560-565. 1999.  M. Slater. How colorful was your day? Why questionnaires cannot assess presence in virtual environments. PresenceTeleoperators and Virtual Environments, 13:484-493. 2004.  Slater, M., Brogni, A., Steed, A. (2003). Physiological Responses to Breaks in Presence: A Pilot Study. Presence 2003: The 6th Annual International Workshop on Presence.  Slater, M and Steed, A. A virtual presence counter. Presence: Teleoperators and Virtual Environments, 9(5): 413–434, 2000.  M. Slater, C. Guger, A. Antley, A. Brogni, D. Friedman, M. Garau, H. Ritter- Widenfeld, and A. Steed. Analysis of Physiological Responses to a Social Situation in an Immersive Virtual Environment, Submitted 2005.  A., Steed, V. Vinayagamoorthy, and A. Brogni, "Using Breaks in Presence to Identify Usability Issues", To be presented at the 11th International Conference on Human Computer Interaction, 22-27, Las Vegas, July 2005.  A. Steed, J. Mortensen, E. Frecon. Spelunking: Experiences using the DIVE System on CAVE-like Platforms. In Immersive Projection Technologies and Virtual Environments. SpringerVerlag/Wien, pp. 153-164, 2001.  M. Thomsen. Discovering Mixed Reality: inventing design criteria for an action-based Mixed Reality, PhD thesis, University College of London, 2004.  M. Usoh, E. Catena, S. Arman, M. Slater. Using presence questionnaires in reality. Presence-Teleoperators and Virtual Environments, 9:497-503, 2000.  D. Watson, R. Friend. Measurement of Social-Evaluative Anxiety. J. of Consulting and Clinical Psychology, 33:448-&, 1969.