Summary: | Networks are becoming an essential component of
modern cyberinfrastructure and
this work describes methods of designing
distributed applications for high-speed networks
to improve application scalability, performance
and capabilities. As the amount of data generated by scientific
applications continues to grow,
to be able to handle and process it,
applications should
be designed to use parallel, distributed
resources and high-speed networks.
For scalable application design developers should
move away from the current component-based
approach and implement instead an integrated,
non-layered architecture where applications
can use specialized low-level interfaces.
The main focus of this research is on interactive,
collaborative visualization of large datasets. This work
describes how a visualization application
can be improved through using distributed resources
and high-speed network links
to interactively visualize
tens of gigabytes of data and
handle terabyte datasets while
maintaining high quality.
The application supports interactive frame rates,
high resolution, collaborative visualization and
sustains remote I/O bandwidths of several Gbps (up to
30 times faster than local I/O).
Motivated by the
distributed visualization application,
this work also researches remote data access systems.
Because wide-area networks may
have a high latency, the remote I/O system uses an architecture
that effectively hides latency.
Five remote data access architectures are analyzed and the results
show that an architecture
that combines bulk and pipeline processing is
the best solution for high-throughput remote
data access. The resulting system, also supporting
high-speed transport protocols and configurable remote
operations, is up
to 400 times faster than a comparable
existing remote data access system.
Transport protocols are compared
to understand which protocol
can best utilize high-speed network connections,
concluding that a rate-based protocol
is the best solution, being 8 times faster than standard TCP.
An HD-based remote teaching application experiment
is conducted, illustrating the potential
of network-aware applications in a production
environment.
Future research areas are presented,
with emphasis on network-aware optimization,
execution and deployment scenarios.
|