|
|
Shared
Virtual Environments
Abstract
Virtual
Reality is a powerful tool for training, simulation, and computer
aided design. The state of the art flight simulators are an
example of the realism that can be achieved. The sensation of
being in a real environment, while interacting with a VR simulation,
is usually referred as sense of presence or sense of immersion.
In most of the current applications the focus has been in providing
a good visual feedback to the user, trying to improve the tracking-registration
and the resolution-refresh rate of head mounted displays. However,
the lack of proprioception, or in other words, the impossibility
of really touching the objects, makes the interaction unreal
and more difficult.
In safety critical VR applications, force feedback is already
provided:
In
the more sophisticated flight simulators force feedback is used
to enhance the feeling of immersion. The operator sits on a
6 DoF parallel manipulator that reproduces part of the flight
dynamics, and holds an active stick, that provides the exact
resistance to motion.
VR
systems offer the possibility of training surgeons without risking
casualties and minimizing the cost. However, force feedback
is indispensable, otherwise the perceptual experience in the
virtual world will not correspond to the real one, making the
training useless, if not dangerous.
In many applications, it is desirable to allow multiple users
to interact together in a shared virtual environments. A platoon
of soldiers, a plane crew, a surgical team can, in a shared
VE, train together.
The level of interaction depends on the physical constraints
of the communication media, the computational power available,
and the characteristics of the haptic displays
The core components of the shared virtual applications we implemented
are:
A
3 Degree of Freedom (DoF) Virtual Environment , in which a single
user can look at and touch objects, using either one of two
haptic displays .
A
Shared Interactions Software Library, to build real-time interactive
applications, such as shared virtual simulations, teleconferencing,
etc.
Using the
virtual simulation package and the shared interaction software,
we implemented three examples of a shared interactive environment,
with and without force feedback:
A video game, Force Feedback, Multi Player Squash , implemented
to test a token-ring architecture for shared virtual environments
with force feedback.
Multiple
users can interact in a 2 DoF Shared Virtual Environment, using
a Mouse or an Haptic Device. Users can simultaneously touch the
same objects, feeling the resulting interaction forces if haptic
devices are used. The system has been developed to test distributed
applications in which strict specifications have to be met to
insure a stable interaction.
A Fast,
Interactive, Multi Users Talk System allows multiple users to
communicate real-time (no video link is provided)
A
3 DoF Virtual Environment
The operator
interacts with a virtual simulation running on a computer. He/She
receives visual feedback from a display and haptic feedback from
a force display or master).
The
quality of the virtual simulation depends both on the particular
mechanical interface and on the equations used to model the dynamics
of the virtual objects. If the equations are a sufficiently good
replica of the real dynamics, the only difference in the feeling
perceived in the real and the virtual manipulation is due to the
fact that in the second case the operator is forced to touch the
environment using a mechanical interface.
In our
system we didn't implement the full objects dynamics. Some of
the simplification we made are:
- It is
not possible to apply moments, therefore objects cannot rotate
and there is no torque feedback to the operator.
- Objects
cannot be deformed, even if it is possible to simulate "spongy"
objects;
- Objects
are modeled as a spring, mass, damper, second order system.
Non-linear friction has also been included.
- Only a
limited number of shapes has been implemented, such as cylinders,
cubes, nuts and regular polygons;
Some data about
the system performance (including SW & HW):
- Max Object
Stiffness that can be simulated = 70000 N/m (HBFD), 3000N/m
(penbased)
- Max Sampling
Rate, including virtual simulation and haptic device interface
= 5 kHz
Shared
Interaction Software Library
This software
provides various tools for the implementation of shared virtual
experiences:
- an enhanced
UDP protocol for communications, built on top of the standard
UDP.
Standard UDP allows a faster round trip communication compared
to the TCP/IP protocol, but packet may be lost or received out
of order. Our enhanced UDP includes in an extended header some
additional information typical of shared simulation.
The implementation allows both
- reliable
transmission, letting the programmer complete freedom to define
the retransmission policy for lost packet, to guarantee a
reliable flow for important single event, that need not to
be lost, such as requests of connection to the shared simulation,
or transmission of one-time data.
- unreliable
transmission, in the case of continuous flow of information,
such as the positions of the haptic devices, for which, the
loss of some packets is not critical.
- heart
beat sending and checking.
- managing
list of connected users.
- a basic
set of procedure to connect, disconnect, synchronize the beginning
of the simulation.
Force
Feedback, Multi Player Squash
We implemented
Force Feedback Multi Player Squash to test a token-ring, or "one
at a time" architecture, for shared virtual environments with
force feedback.
Integrating
a force feedback interface in a shared virtual environment (SVE)
introduces some serious constraints. If multiple users are to
manipulate the same portion of the SVE, demanding requirements
have to be satisfied for latency and sampling rate, as we explained
in the previous paragraphs. If minimum latency cannot be guaranteed,
because of the characteristics of the communication media, a compromise
has to be accepted, otherwise the system might be unstable, and
therefore dangerous to the user.
One
possible solution is a "one-user-at-a-time" architecture. Only
one user at a time, the Active Operator (AO), is allowed to modify
and feel force from the environment. The other users, non-AO,
are allowed to watch the on going interaction through a visual
display, move around in the SVE, but not touch or modify objects.
A copy of the environment run on each local machine. Once the
AO has completed his/her interaction, all the copies of the SVE
are updated to match the AO's copy. The AO then releases the role,
that is assigned by the scheduler to a different operator.
The
system could be implemented with a sophisticated scheduler, that
allows different AO in different area of the virtual object (as
in a database, for different items), or in a more simple implementation,
the operators could share the role of AO in a token ring fashion.
The role of AO is passed from player to player, in a fixed order.
In FFMS,
more than two players can play together, but as in real squash,
only the AO is allowed to touch the ball. Using a haptic device,
as in real squash, players feel the collision with the ball. The
speed of the ball after collision, and the intensity of the stroke
reflected to the operator, is determined by applying the principle
of conservation of energy to the system composed of ball and racket.
The
system architecture is peer-to-peer like, in other words, information
is not only sent to a central node (the server) but exchanged
between all clients (see Figure).
The
system consists of:
- a Server,
that is used to handle connection-disconnection requests to
the game, to synchronize the start and end of the game, and
to monitor the correct functioning of the system;
- multiple
Clients, one per player. Each Client contains the complete model
of the system, that consists of the dimensions of the squash-court,
the position of all players, and the position and the speed
of the ball.
All players
periodically (at a changeable frequency, set by default to 20Hz)
send their relative position, plus the position of the ball if they
are the AO, to the others players and the server. For a medium-large
number of players, instead of actually sending the same packet to
all the others players, a single packet could be sent using multi
casting.
Non-AO update
the position of the ball, in between reception of new packets,
using a predictive model. Once a new packet arrives, the new data
is used to adjust the estimated position. This technique is called
dead-reckoning.
We implemented
an Adaptive-Dynamic-Bounder, to ensure that the "dynamics" of
the simulation is compatible with the delay in the communication.
To this purpose, each client estimates the round trip delay in
between peers, and sends the data to the server. The server determines
the maximum round trip delay, and sends back to all the clients
what are the maximum allowed absolute speed and maximum change
in speed, after a collision of the ball with a virtual racket.
A similar technique, not yet implemented, could be used to limit
the maximum speed achievable by the user in moving the racket,
introducing an active damping on the haptic display proportional
to the maximum delay. In such a way, not only the dynamics of
the ball could be limited, but also that of the players.
The protocol
used for communication is the enhanced UDP described in the Shared
Interactions Software Library.
Devices
Pen
Based Force Display (PBFD)
Publications
(*)
(*)
Note: Most of the BRL
publications are available on-line in a PDF format.
You may used the publication's reference number as a link to the
individual manuscript.
[076]
P. Buttolo, D.Y. Hwang, B. Hannaford,
'Hard Disk Actuators for Mini-Teleoperation,' Proc. SPIE
Telemanipulator and Telepresence Technologies Symposium, pp. 55-61,
Boston, October 31, 1994.
[081]
P. Buttolo, P. Braathen, B. Hannaford,
'Sliding Control of Force Reflecting Teleoperation: Preliminary
Studies,' PRESENCE, vol. 3, pp. 158-172, 1994.
[089]
P. Buttolo, B. Hannaford,
'Pen Based Force Display for Precision Manipulation of Virtual
Environments,' Proceedings VRAIS-95, pp. 217-225, Raleigh,
NC, March 1995.
[093]
B. Hannaford, P.H. Marbot, P. Buttolo, M. Moreyra, S. Venema,
'Scaling Properties of Direct Drive Serial Arms,' International
Journal of Robotics Research, vol. 15, pp. 459-472, 1996.
[094]
P. Buttolo, D. Kung, B. Hannaford,
'Manipulation in Real, Virtual, and Remote Environments,'
Video Proceedings, IEEE VRAIS-95, Raleigh, NC, March 1995.
[096]
P. Buttolo, D. Kung, B. Hannaford,
'Manipulation in Real, Virtual, and Remote Environments,'
Proceedings, IEEE International Conference on Systems, Man, and
Cybernetics, Vancouver, BC, October, 1995.
[097]
B. Hannaford, A.K. Bejczy, P. Buttolo, M. Moreyra, S. Venema,
'Mini-Teleoperation Technology for Space Research,' Procedings
International Symposium on Microsystems Intelligent Materials
and Robots (MIMR-95), pp. 524-527, Sendai, Japan, September, 1995.
[098]
P. Buttolo, B. Hannaford,
'Advantages of Actuation Redundancy for the Design of Haptic Displays,'
Proceedings, ASME Fourth Annual Symposium on Haptic Interfaces
for Virtual Environment and Teleoperator Systems, vol. DSC-57-2,
pp. 623-630, San Francisco, Nov. 1995.
[101]
P. Buttolo, R. Oboe, B. Hannaford, W. McNeely,
'Force Feedback in Shared Virtual Simulations,' Proceedings
MICAD, Paris, 1996.
[112]
P. Buttolo, R. Oboe, B. Hannaford,
'Architectures for Shared Haptic Virtual Environments,' Computers
and Graphics, vol. 21, pp. 421-9, July-Aug 1997.
[119]
P. Buttolo, B. Hannaford,
'Direct Drive Manipulator for pen-based force display,' U.S.
Patent #5,642,469, June 24, 1997 (issued).
[M0054]
P. Buttolo, J. Hewitt, R. Oboe, B. Hannaford,
'Force Feedback in Virtual and Shared Environments,' Poster:
IEEE Intl. conf. on System, Man and Cybernetics, Vancouver BC,
October 1995.
[Th016]
P. Buttolo,
'Characterization of Human Pen Grasp with Haptic Displays,'
Ph.D. Dissertation, University of Washington, Department of Electrical
Engineering, June 1996.
|
|
|
