Distributed and Parallel Computing (eBook)

Sandhya Avasthi, Suman Lata Tripathi (Herausgeber)

eBook Download: EPUB

2025
524 Seiten
Wiley-Scrivener (Verlag)
978-1-394-28801-4 (ISBN)

Infos

Master the growing field of distributed and parallel computing with this essential guide, offering expert insights into the fundamentals and real-world applications for intelligent and collaborative systems.

Distributed computing, or running programs across multiple computers over a network, is becoming a popular solution for addressing the demands for increased performance across industries, including scientific computing, oil exploration, biotechnology, and medicine. Distributed computing enables seamless communication and collaboration by allowing users from different locations to access and interact with their digital twin simultaneously. Distributed computing enhances the capabilities of digital twins by providing scalability, parallel processing, real-time data integration, collaboration support, resource optimization, fault tolerance, and security features. Distributed and Parallel Computing explores the fundamentals and innovations in intelligent and distributed computing systems and applications, including adaptivity and learning, agents and multi-agent systems, argumentation, case-based reasoning, and collaborative systems. Through expert insights, readers will discover promising real-world applications for this emerging technology.

Sandhya Avasthi, PhD is an assistant professor in the Computer Science and Engineering Department at ABES Engineering College at Dr. Abdul Kalam Technical University with over 18 years of teaching experience. She has published numerous research articles in refereed international journals, conference proceedings, and book chapters. Her research interests include natural language processing, information extraction, information retrieval, data science, and business intelligence.

Suman Lata Tripathi, PhD is a professor at Lovely Professional University with more than 22 years of experience in academics and research. She has published more than 19 books, 125 research papers in refereed science journals, conference proceedings, 13 Indian patents, and four copyrights. Her area of expertise includes microelectronics device modeling and characterization, low-power VLSI circuit design, VLSI design testing, and advanced FET design for IoT.

Textauszug

1
Introduction to Distributed Systems

K. Karthikeyan1*, S. Hemalatha2 and S. Vignesh1

1Department of Computer Science and Business Systems, K.S. Rangasamy College of Technology, Tiruchengode, India

2Department of Computer Science and Design, Kongu Engineering College, Perundurai, India

Abstract

A distributed system is an assembly of independent computers linked by a network that cooperates to accomplish a shared objective. Distributed systems, as opposed to conventional centralized systems, which assign all work to one machine, provide for greater performance, fault tolerance, and scalability. The goal of communication-efficient distributed machine learning is to train machine learning models across several nodes while minimizing the communication overhead in distributed systems. In distributed machine learning, data are distributed across various nodes, and computation is performed in parallel. However, exchanging data and model parameters between nodes can lead to significant communication costs, especially in scenarios with large datasets or complex models. Training complex machine learning models on large datasets is very computationally intensive. Distributed ML systems partition the data and workload across multiple machines to parallelize and speed up training. However, the communication cost of synchronizing model parameters can dominate system performance. Blockchain technology is inherently based on distributed systems principles. It decentralizes the storage and management of data, ensuring transparency, security, and reliability. The legitimate network state and history must be agreed upon by all nodes running the blockchain software. To establish the canonical chain, consensus techniques including proof-of-work, proof-of-stake, and Raft are employed. By agreement, fault tolerance is provided. Each node in the blockchain network keeps a local copy of the whole ledger or state. Transparency is enabled by many mechanisms that broadcast and replicate changes among nodes. Trends such as serverless computing, edge network growth, distributed ledgers, and autonomic systems point to more diffusion of distribution across the computing stack. For sectors like healthcare, distributed technologies promise to transform care delivery by connecting data and providers across geographic and organizational silos. Distributed systems have clearly emerged as a transformational paradigm in modern computing. But truly realizing their potential while managing their complexities continues to be an active domain for innovations in algorithms, mathematical models, systemic architectures, and practical tools.

Keywords: Distributed, blockchain, Internet of Things (IoT), raft, machine learning, nodes, healthcare

1.1 Introduction

A group of separate computers that appear to users to be a single, cohesive system is known as a distributed system. These systems are designed to share resources and coordinate their actions to achieve a common goal. The computers in a distributed system are connected through a communication network and communicate with each other by passing messages.

1.1.1 Background and Context

The primary motivation behind distributed systems is to achieve better performance, scalability, fault tolerance, and resource sharing. By distributing the workload across multiple computers, distributed systems can handle larger computational tasks and support more users than a single computer system. Additionally, distributed systems can provide redundancy, ensuring that if one component fails, the system as a whole can continue to operate.

Some key characteristics and challenges of distributed systems include:

Concurrency: Multiple components in a distributed system may execute concurrently, which can lead to synchronization issues and data inconsistencies.

Lack of a global clock: It is difficult to maintain a consistent global clock across distributed components, which can make it challenging to order events consistently.

Scalability: Distributed systems should be able to scale up or down by adding or removing components without significantly affecting the overall performance or functionality.

Transparency: Ideally, the distribution of resources and components should be transparent to users, who should perceive the system as a single, cohesive unit.

Heterogeneity: Numerous hardware and software components are frequently used in distributed systems. Heterogeneity, coming from several vendors, may provide compatibility and integration issues [1].

Figure 1.1 shows the evidence of distributed systems are widely used in various domains, including web applications, cloud computing, peerto-peer (P2P) networks, big data processing, and distributed databases. Examples of popular distributed systems include Apache Hadoop and Kubernetes. To design and implement distributed systems, developers often rely on architectural patterns and principles, such as the client-server model, message queues, load balancing, replication, and consensus algorithms (e.g., Paxos, Raft). Additionally, distributed systems may leverage technologies such as remote procedure calls (RPCs), distributed file systems, and distributed caching to facilitate communication and data sharing among components.

A common diagram used to illustrate the concept of a distributed system would be a network diagram, depicting multiple computers or nodes connected through a communication network.

• Numerous hardware and software components are frequently used in distributed systems. Heterogeneity, coming from several vendors, may provide compatibility and integration issues.
• The communication network that allows these nodes to communicate with one another is represented by the lines joining them, which might be the Internet, a local area network (LAN), or any other appropriate network.
• The nodes are independent and autonomous but can share resources and coordinate their actions by sending and receiving messages over the network.
• The diagram shows how distributed systems are decentralized, meaning that several nodes cooperate to accomplish a particular objective rather than depending on a single hub for failure or control.

Figure 1.1 Distributed systems.

This is a simplified representation, and real-world distributed systems can be much more complex, involving different types of nodes (e.g., servers, clients, load balancers, databases), different network topologies, and various communication protocols and technologies [2].

1.1.2 Objectives of the Study

The intrinsic difficulties and demands of distributed systems motivate a number of important goals that the study of these systems seeks to accomplish. The goal of studying distributed systems is to develop scalable, secure, dependable, and efficient systems that can meet the ever-increasing needs of contemporary computer settings. These goals can be met by researchers and developers to build reliable and efficient distributed systems that support a variety of services and applications.

1.1.3 Scope and Limitations

Distributed systems have a broad scope, enabling large-scale computing tasks, high availability, fault tolerance, efficient resource sharing, load balancing, scalability, geographic distribution, and interoperability across heterogeneous environments. They are essential in areas such as the Internet of Things (IoT) and enable parallel computing. However, distributed systems also face several limitations. Their complexity arises from synchronization requirements, fault tolerance mechanisms, and coordination among multiple nodes. Network latency, bandwidth constraints, and maintaining data consistency and integrity across nodes pose challenges. Security concerns related to data privacy, integrity, and access control are heightened due to communication and data sharing among nodes. Integrating diverse hardware and software components can lead to compatibility issues. Debugging, troubleshooting, and handling partial failures become more difficult compared with centralized systems. Efficient resource allocation and load balancing across nodes require sophisticated algorithms. While offering numerous benefits, the inherent complexity and potential limitations of distributed systems must be carefully considered during design, implementation, and operation to mitigate risks and maximize their advantages as mentioned in Table 1.1.

Table 1.1 Scope and limitations of distributed systems.

SN.	Scope	Limitations

1

Large-scale computing

Complexity

2

High availability and fault tolerance

Network latency and bandwidth

3

Resource sharing and load balancing

Consistency and data integrity

4

Scalability

Security and trust

5

Geographic distribution

Heterogeneity challenges

6

Heterogeneous environments

Debugging and troubleshooting

7

Parallel processing

Partial failure handling

8

Internet of Things (IoT)

Resource allocation and load balancing

1.1.4 Key Characteristics of Distributed...

Produkt-Details

• Erscheint lt. Verlag: 10.11.2025
• Sprache: englisch
• Themenwelt: Mathematik / Informatik ► Informatik ► Netzwerke
• Themenwelt: Mathematik / Informatik ► Informatik ► Theorie / Studium
• Schlagworte: Cluster Computing • Digital Workplace • distributed algorithms • Distributed Digital Twin • Distributed Intelligent Sensing • Distributed Model • distributed processing • Distributed System in 6G • Distributed Systems • High-Performance Computing • Industry 4.0 • Parallel Algorithms • Parallel Computing • parallel machines • Parallel Random-Access Machines
• ISBN-10: 1-394-28801-8 / 1394288018
• ISBN-13: 978-1-394-28801-4 / 9781394288014