 |
THE AIMS OF THE PROJECT
In his 1965 book, Understanding Media, Marshall McLuhan wrote, "The
hybrid or meeting of two media is a moment of truth and revelation of which
new form is born...The crossings of media release great force." In
many ways, this sort of bringing together is exactly what researchers have
done with virtual reality. The technologies of computer graphics, data communications,
and computer programming have been loosely melded with the technologies
of the telephone, the television, and the video game. The result is a single,
ever-changing technology, immensely superior to each of its predecessors.
Although the technology is mature enough to have different applications,
there are key issues to be resolved for its use for medical applications.
- The product seem to be "a solution in search of a problem".
As with early computer graphics products, the entry-level costs are relatively
prohibitive. A complete VR environment, including workstations, goggles,
body suits, and software, is in the range of KEcu 70.000 to KEcu 1.000.000.
- The hyperbole and sensational press coverage associated with some of
this technology has led many potential users to overestimate the actual
capabilities of existing systems. Many of them must actually develop the
technology significantly for their specific tasks. Unless their expertise
includes knowledge of the human-machine interface requirements for their
application, their resulting product will rarely get beyond a "conceptual
demo" that lacks practical utility.
- The premise of VE seems to be to enhance the interaction between people
and their systems. It thus becomes very important to understand how people
perceive and interpret events in their environments, both in and out of
virtual representation of reality. We must address issues of human performance
to understand how to develop and implement VE technology that people can
use comfortably and effectively. Fundamental question remain about how
people interact with the systems, how they may be used to enhance and augment
cognitive performance in such environments, and how they can best be employed
for instruction, training, assessment, rehabilitation and other people
oriented applications.
1. Objectives of the project
2. Key issues of the VREPAR project
3. A framework for future researches in
VR human issues |
|
1. Objectives of the project
It is precisely one of the goals of the project to overcome those shortcomings
in order to develop a demonstrator of a virtual environment that can be
effectively used by end users. The main goal of the project is to build
a demonstrator of a virtual reality system (Virtual Reality All-purpose
Modular System - VRAMS), based on a modular architecture, to be used for
psychoneurophysiological assessment and rehabilitation.
Various studies aimed at the development of virtual reality systems for
this type of application have been under way for some time now, but the
existing systems have a series of problems that limit their real possibilities
of use:
- Cost: Although some attempts have been made to use PC-based
virtual reality systems, the majority of the existing systems use RISC
platforms whose cost is beyond the means of a normal Hospital Centre or
Department.
- Lack of reference standards: Almost all of the applications
in this sector can be considered "one-off" creations tied to
their development hardware and software, which have been adjusted in the
field by a process of trial and error. This makes them difficult to use
in contexts other than those in which they were developed.
- Non-interoperability of the systems: Although it is theoretically
possible to use a single virtual reality system in many different applications,
none of the existing systems can be easily adapted to different tasks.
This means that two different wards/departments within the same organisation
may find themselves having to use two different VR systems because of the
impossibility of adapting one single system to their different needs.
- Lack of network connections: The possibility of establishing
network connections could considerably increase the ways of using the systems
by introducing their possible remote use but, with the exception of a few
prototypes, both hardware and software problems make it difficult to network
the existing systems.
With the development of VRAMS, VREPAR has the following objectives:
a) To develop a virtual reality system for the medical market that can
be marketed at a price which is accessible to its possible end-users (hospitals,
universities and research centres) and which would have the modular, connectability
and interoperability characteristics that the existing systems lack;
b) To develop three hardware/software modules for the application of
VRAMS in psychoneurophysiological assessment and rehabilitation. The chosen
development areas are eating disorders (bulimia, anorexia and obesity),
movement disorders (Parkinson's disease and torsion dystonia) and stroke
disorders (unilateral neglect and hemiparesis).
c) To define reference standards and parameters relating to technological
and experimental factors, which can also be used by third parties. In particular,
VREPAR aims at:
- defining a standard for software development and a series of hardware
specifications to be used in the development of further VRAMS modules;
- identifying the factors affecting individual experiences in a virtual
environment;
2. Key issues for the VREPAR project
Undoubtedly the construction of different VR modules brings out many
interesting and challenging design questions. The following key points have
emerged from our survey of PC related market that are relevant for VREPAR
Project's aims:
- PC hardware market is chaotic and restless, thus compelling
not to bet on particular architectures but only on converging tools while
waiting for wider standardisation;
- Virtual Reality building software packages differ in
terms of hardware support and of implemented standard formats;
- Each market segment has its peculiar solutions and "best
buys", hardly replaceable by different choices;
- A convergence is necessary between these different areas
in order to achieve a sort of "work-in-progress" standard, based
upon the common support of interfacing tools;
- VRML language and world building/browsing tools do provide
such chance of convergence.
A suitable approach is in considering these points not as constraints
for our research but to turn them into advantages for the successful outcome
of the project. The instability of the market can drive us towards the adoption
of a standard highly inter-operable and multi-platform like VRML. It may
be widely diffusable because almost all VR packages support the possibility
of importing/converting VRML worlds. In terms of low cost technology it
satisfies the requirements under the most convincing circumstances because
a VRML application is unbounded from platform specifications and available
to immediate exploration in the Internet which we can consider under this
perspective the most promising method for low-cost VR. The Internet exploitation
opens a wide set of possibility linked to the envisaging of medical practices
as activities freed from time and location constraints. VR could assist
highly demanded operations such as tele-diagnostics, tele-therapy and many
others both as one-to one or as collaborative processes among different
roles and actors opening exciting new perspectives for medical VR.
But the problem of VREPAR is not just a technical problem of assembling
highly technical devices, it is also a problem of rendering the efficacy
and successful outcome of the project with the intrinsic guarantee that
any VR interaction treatment whether be diagnostic or therapeutic must definitely
be not harmful and cause any danger to patients and users. Like for any
other medical aim a specific concern on the absence of collateral effects
and accurate testing on the result of somministration is mandatory due to
the high responsibility of dealing primarily with human health. There is
a literature on the effects that VR interaction may cause to users, these
effects while being limited to special sectors of population can appear
and undermine the clinical benefit with unpredictable and unwanted results.
So in this perspective it has to be considered one of the key issues an
accurate and attentive consideration of human factors involved, this has
to be achieved at this stage of the project were hardware software characteristics
must be able to set their use to the mandatory concern of human health and
safety.
Professional literature on Human Computer Interaction - HCI specifically
requires from the prototyping phase special consideration of these issues
which are not ignored in the project and will be attentively treated as
the work will progress.
Another consideration can be made for what concerns the more accurate
deepening and tuning of the different modules to the specific cognitive
and social contexts of use.
3. A framework for future researches in VR human
issues
As underlined before, the problem of VREPAR is not just a technical problem
of assembling highly technical devices. A main aim of the project is to
assure that VR interaction treatment, whether be diagnostic or therapeutic,
is not harmful and cause any danger to patients and users. To establish
a framework for the future research in VR human issues, a method is needed
to quantify the temporal performance of virtual reality systems, ensuring
the HCS characteristics can cope with them since the prototyping phase.
Infact, as we seen in the human factor part, many of the undesirable
effects of exposures to virtual reality environments can be ascribed to
limitations in the temporal performance of the system hardware. These limitations
result in lags between movements of the head or hands and corresponding
movements of displayed images.
A mathematical model is described below which could provide the basis
for such a framework. The inputs to the model are either the angular displacements
of tracked objects, or recorded head motions (Lewis and Griffin, 1996).
3.1 Input output models of the temporal performance
of the VR systems
3.2 Applications of systems models
3.3 Recommendation for the design of
applications
3.1 Input-output models of the temporal performance of virtual reality
systems
Transfer function models are used to represent the sampling delays in
a head position sensor, image processor and display. Transfer function models
can also be used to represent the tracking behaviour of users and their
vestibular and ocular responses to head and image motions (Lewis and Griffin,
1996). The dynamic of the various components of the model are discussed
in below.
The transfer functions of the system components can be combined into
the input-output model shown in Figure 2. The primary inputs to the model
are the displacements of the visual target. A head motion time history may
also be input directly to the model, in which case the elements of the human
operator tracking model, Hh(s) and R(f) would be
set to zero.
3.1.1 Modelling the temporal characteristics of the human
operator
The head tracking response
The human operator in a continuous tracking task can be modelled as a
linear transfer function, Hh(f), where:
where Oh(f) and I(f) are frequency domain representations
of the head position output, oh(t), and the target position
input, i(t). From an engineering point of view a human operator is
a non-linear control element, but Krendel and McRuer (1965) showed that,
with particular inputs, subjects responded similarly to equivalent linear
systems. However, to completely describe the system output it was necessary
to include a "remnant" component, r(t). The remnant comprises
a relatively small signal which is added to the output of the linear model
to account for differences between the response of the operator and the
time-invariant linear relationship between the system input and both the
tracking error and the system output. This type of model is referred to
as a quasi-linear model of the human operator. The remnant is modelled as
normally distributed broad-band random noise.
So and Griffin (1995a) have measured closed-loop transfer functions for
head-aiming. A cross-hair aiming reticle was displayed in the centre of
a monocular helmet mounted display. A circular target was driven by independent
random functions in both the pitch and the yaw axes. The head coupled system
had an inherent lag of approximately 40 ms between head movements and the
corresponding movement on the display (comprising the lag in the head position
sensor, the computation time and the update rate of the display). An additional
time delay, which was varied between 0 ms and 160 ms, was imposed by a computer.
Figure 3 shows mean head tracking transfer functions in the pitch and
yaw axes measured with five values of imposed system lag. At frequencies
below 0.4 Hz the modulus (i.e. the gain) of the human operator transfer
functions were all close to unity but at higher frequencies the gain increased
with increasing imposed display lag. The increased gain at higher frequencies
was believed to be a strategy used by the subjects to compensate for the
increased lag in the system. The phase lags reflect delays in the operator's
response, relative to the target motion. The phase lags at around 0.1 Hz
were shown to decrease significantly with increasing lag, but the reduction
was not sufficient to compensate for the imposed display lag. The phase
response at 0.1 Hz was important since a large proportion of the energy
in the target motion was around this frequency. Although there was a decrease
in the phase lag of the subjects with increasing system lag at low frequencies,
the phase lags above 0.5 Hz increased with increasing system lag. This is
a typical response for a system consisting of a lead-lag filter and a lag:
lead generation at low frequencies is accompanied by increases in response
lag at higher frequencies, and a consequent reduction in tracking bandwidth
(McRuer, 1973).
The vestibulo-ocular reflex
The vestibulo-ocular reflex (VOR) induces movements of the eyes which
compensate for movements of the head. The VOR stabilises the eyes in space
during body motion. This makes it possible to view objects which are fixed
in space without smearing of the retinal image. The frequency response characteristics
of the compensatory eye movements induced by angular oscillation in yaw
have been measured by, for example, Benson and Barnes (1978). Barnes (1980)
has modelled the slow phase (compensatory) eye movements evoked by yaw axis
rotation by the following transfer function:
where s is the complex radial frequency; is the angular displacement
of the eye; is the angular displacement of the head; T=0.005 s; TA=0.2
s; TB=15 s; TC= 0.125 s; TD=
0.002 s; TAD=80 s and KC=0.7.
The pursuit reflex
The VOR can be overridden at low frequencies by a pursuit tracking reflex.
Benson and Barnes (1978) have discussed methods for modelling this visually-driven
response. The pursuit reflex has been shown to break down when the velocity
of a viewed object is greater than 40 to 60 degrees per second or the frequency
of the movement is above about 1 Hz (Benson and Barnes, 1978).
3.1.2 Modelling the temporal characteristics of system
components
Head position sensor
Typical head pointing systems sample the angular displacement of the
head with sample rates between 30 and 60 samples per second. The output
signal of the head pointing system, (s), in response to head displacement,
(s), can be represented by:
where *(s) is the signal (s) sampled at discrete intervals
of T seconds and is given by:
If the system operates at a sampling rate of 30 samples per second, T
= 2/s = 0.033 s. Hdelay(s) represents
a time delay of T seconds and is equivalent to:
Hhold(s) is a zero-order hold and is equivalent to:
Image processing
The image processor is a computer which takes the head angle measured
by the head pointing system and renders the images which are displayed on
the head-mounted display at appropriate locations in the video frame. The
computation will impose a time delay and a sample-and-hold effect. Hence
the transfer function, Hc(s), has the same form as that
of the head pointing system. The computation time, T, is the inverse
of the system frame rate.
Head-mounted display
The response and frame rate of the display combine to produce a sample-and-hold
effect, hence the transfer function representing the effect of the display
on the position of a moving image will be approximated by:
with a frame rate of 60 frames per second T = 2/s
= 0.0167 s. Hhold(s) is a zero-
order hold given by:
Predictive filter
The characteristics and implementation of phase-lead filters for predicting
future head motions have been discussed by So (1995) and So and Griffin
(1996). Alternative methods are cited by Wioka (1995). Alternative head
position prediction algorithms may be implemented in the model to determine
the extent to which they reduce the effects of system lags.
3.2 Application of system models
Evaluation of visual-vestibular interaction
Perception of rotational self-motion may be examined using models such
as that proposed by Zacharias and Young (1981). The inputs to this model
are the head velocities sensed by the semi-circular canals, ves,
and by peripheral vision (i.e. the world-stabilised background image), vis.
the extent of potential cue conflicts between visual and vestibular perception
of self-motion may be derived from the difference between the two signals.
Evaluation of visual-vestibular interaction in the perception
of angular self-motion.
This figure shows how that generic model may be adapted to evaluate potential
cue conflicts. The figure also shows the velocity difference between the
head velocity and image velocity time histories. In this example the system
update rate was 10 Hz and the lag in the head pointing system was 15 ms.
A conflict index was derived from the relative magnitude of the velocity
difference and the head velocity signal. The conflict index was calculated
from:
With the fast head motion the system lag induces a relatively large velocity
difference, resulting in a conflict index of 0.81. With the slower, continuous
head motions the velocity difference is smaller compared with the head velocity,
resulting in a conflict index of 0.19.
The above example illustrates the importance of the interaction between
the system lags in an immersive virtual environment and the characteristics
of the head movements made by the users. The head motion characteristics
need to be taken into account when optimising the system for a particular
application.
Registration errors
The model can be used to evaluate dynamic errors in image registration,
between the real world seen through a head-mounted display, and overlaid
virtual objects. The figure below shows how the generic model may be adapted
to evaluate image registration errors in augmented velocity tasks.
Evaluation of tracking performance and registration errors.
3.3 Recommendations for the design of applications
It might be possible to reduce side effects of exposures to virtual reality
environments by either optimising the design of the system (i.e. the software
and the hardware) or by implementing procedures to manage exposures. Recommendations
for managing exposures to flight simulators and virtual reality environments
have previously been made by Frank et al (1983) and McCauley and
Sharkey (1992).
The users of virtual reality applications designed for assessment and
rehabilitation may have disabilities which increase their susceptibility
to certain side-effects. Many of the reported effects on performance and
well-being have been ascribed to distortions in the relationship between
the movements of users, and the visual feedback of those movements. These
distortions arise because of limitations in the spatial and temporal performance
of virtual reality systems, or because image motions are presented which
give the illusion of body motion in the absence of real motion.
The proposed VRAMS modules (at least the Eating Disorders one) will deliberately
create further distortions in order to augment the stimulation and feedback
provided by the visual images. These distortions may be expected to give
rise to additional problems. Special precautions therefore need to be taken
to ensure the safety and effectiveness of such virtual reality applications.
Section 4.3.4.1 makes specific recommendations for managing exposures to
virtual reality environments for assessment and rehabilitation.
The design of virtual reality applications for assessment and rehabilitation
may also require special precautions to ensure the effectiveness of the
tasks presented to subjects. Section 4.3.4.2 makes design recommendations
based on current knowledge of the effects of system characteristics on the
performance of tasks and the incidence of side-effects.
3.3.1 Managing exposures to virtual reality environments
- Subjects should be informed of the possible adverse effects of exposure
to virtual reality environments before they are exposed. They should be
encouraged to report symptoms as soon as they occur. They should also be
warned about the possible delayed onset of symptoms.
- The state of health of subjects should be screened before they are
exposed to immersive or partially immersive virtual reality environments.
Subjects should not be exposed if they are suffering from illnesses such
as flu and ear infection, sleep loss or the effects of alcohol. Any medication
subjects are taking should be reviewed for possible effects on visual or
vestibular function. The postural stability of disabled subjects should
also be evaluated to determine whether special precautions may need to
be taken to prevent falling during or after exposures. Subjects may also
be screened by a motion sickness history questionnaire (Kennedy et al,
1992) to make the experimenter aware of individuals who may have a high
risk of developing simulator sickness symptoms.
- Exposure duration to immersive or partially immersive virtual reality
environments should be limited to less than 5 minutes at the initial exposures.
The duration of subsequent exposures may be gradually increased as the
subject adapts to the virtual reality environment. Subjects should be carefully
monitored during the adaptation process for simulator sickness symptoms,
and exposures should be terminated when any symptoms become evident.
- The incidence of simulator sickness symptoms in a particular system,
and the exposure time up to the appearance of symptoms, should be routinely
logged. The incidence rate for different categories of symptoms should
be regularly reviewed (Kennedy et al, 1987). This will provide an
assessment of potential problem areas and allow early application of appropriate
counter-measures.
Subjects should be advised to allow a time for recovery after exposure
to partially immersive or immersive virtual reality environments before
engaging in potentially dangerous activities such as driving a car. Individuals
experiencing postural disturbances should be monitored until symptoms have
dissipated. In this case, potentially hazardous activities such as driving
should be restricted for 12 to 24 hours.
3.3.2 Recommendations for the design of the VREPAR modules
The VRAMS modules that will be developed within the VREPAR project all
involve some degree of manual control, or manipulation of virtual objects.
The eating disorders module is a fully immersive virtual reality application
in which users move through a virtual environment, observing and interacting
with virtual objects. The stroke disorders module may utilise a fixed wide
field-of-view display to present moving scenes which could induce a strong
sense of vection. Some special considerations which should be given to the
implementation of these three features are detailed below.
Interactive, immersive virtual environments
- The system should minimise the total system lag between head position
sensing and presentation of an appropriate image. This may involve the
selection of suitable position sensors (Meyer et al, 1992) and the
minimisation of synchronisation errors (Wioka, 1995). If the total lag
cannot be reduced below 100 ms then consideration should be given to implementing
a predictive algorithm for head position in order to reduce the effects
of the lag (Wioka, 1995).
- The application should be designed to require the minimum of head movements.
This is particularly important during initial exposures, before the user
has adapted to the altered environment.
- The maximum speed of self-movement through an immersive virtual environment
should be limited to a very slow rate if it is not initiated by natural
means such as walking.
- Unusual and extraordinary manoeuvres, such as abruptly freezing the
simulation or resetting the scene forward or backward in time, should be
avoided. The scene should be removed altogether for entering and exiting
the virtual environment.
Manual control or manipulation of virtual objects
- The system should be designed to have update rates of no less than
10 frames per second. With lower frame rates, subjects may adopt unnatural
movement strategies, resulting in possible negative transfer of training.
- In applications where the performance of positioning tasks is critical
(i.e. there is a requirement for accurate tracking or the capture of virtual
objects), the total lag between limb movement and the appropriate image
movement on the display should be less than 100 ms. While this is the normal
design requirement it is recognised that the simulation may require lags
and other distortions to be imposed on image movements. The possible effects
of those manipulations require consideration (see Sections 3 and 4).
Presentation of large moving scenes in the absence of
body motion
- Moving images sensed by peripheral vision can induce a strong sense
of vection, and symptoms of visually-induced motion sickness. The strength
of the stimulus depends the velocity of the image motion, and may be reduced
if the subject fixates on fixed objects, or has a degree of control over
the image motion (see Section 4.2.3.2). Particular attention should be
given to the requirement to restrict the duration of initial exposures
and encourage adaptation to the virtual environment.
- Disabled or unsteady individuals should be provided with adequate support
to ensure they are not at risk from falling, since vection may induce inappropriate
postural re-adjustments.
Last Updated: November, 15, 1996
For
any questions or requests, please contact auxo.psylab@auxologico.inet.it