Storage and Redundancy

IPFS (InterPlanetary File System)

IPFS ensures redundancy by distributing copies of files across a decentralized, peer-to-peer network. IPFS breaks down data into smaller chunks and stores it across multiple nodes, making retrieval faster and more censorship-resistant. It assigns each file a unique content-based hash, which allows users to retrieve the file from any node with a copy, thus eliminating single points of failure. If one node goes down, the data remains accessible from other nodes holding the same content. IPFS is essential in redundancy because it provides persistent data access in decentralized applications, making it resilient to data loss and outages while ensuring availability.

Protocol overview (a closer look into IPFS)

IPFS is a revolutionary peer-to-peer (P2P) decentralized protocol designed for content storage and data sharing across a distributed network. Unlike traditional HTTP, which retrieves files based on location, IPFS employs content addressing, allowing files to be located based on their unique content identifiers. This ensures data integrity, immutability, and efficiency in storage and retrieval.

By leveraging advanced technologies such as Conflict-Free Replicated Data Types (CRDTs), Distributed Hash Tables (DHTs), and extrapolated versioning, IPFS provides a robust framework for managing and accessing data. CRDTs help manage files distributed across nodes without conflicts, even when files are updated simultaneously in different locations. At the same time, DHTs allow fast, decentralized file location by storing metadata that quickly points to the most relevant nodes, which hold a copy of any requested file.

This way, IPFS supports immutable and traceable file storage, making it ideal for applications requiring transparency and resilience.

For all these reasons, IPFS is often described as a decentralized HTTP protocol that ensures a secure, scalable, and resilient network capable of handling the challenges of large-scale data distribution across industries.

Three Core Features

IPFS (InterPlanetary File System) supports three core features:

1. Immutability

IPFS ensures data can't be changed or corrupted over time. Data integrity and immutability are achieved by leveraging content-addressing and redundancy (multiple copies of files across nodes). Each file is assigned a unique Content Identifier (CID) based on its content. If a file is modified, the CID changes, ensuring the original file remains intact while the modified version receives a new identifier. This mechanism guarantees that files cannot be silently altered or overwritten.

This behavior forms the basis of what can be called extrapolated versioning. To create a structured versioning system, additional logic or metadata must be applied, associating new CIDs with previous ones to indicate they are versions of the same file.

Systems like the Decentralized Encrypted File System build on this principle by adding an organizational layer. DEFS enables explicit version tracking, linking new versions to their predecessors, and storing metadata such as version numbers or timestamps. For example, if a document is updated, DEFS can automatically label the new CID as "Version 2" of the original file, allowing users to retrieve and compare historical versions effortlessly.

2. Traceability

DEFS further enhances traceability, extending IPFS's capabilities to include a clear history of changes and edits. While IPFS retains all file versions, DEFS organizes this data by associating each version with its predecessor and recording details such as modification timestamps and contributors.

For example, in collaborative environments, DEFS allows users to track who made changes, when they occurred, and what was altered. This is achieved by layering metadata onto IPFS's immutable framework, creating a structured, transparent history of modifications. With IPFS's immutability, DEFS provides a powerful and unique tool in decentralized systems.

3. File Access Efficiency

IPFS makes files easier to access by using content addressing (locating files by their hash rather than their location). Content addressing in IPFS means that files are located based on their content rather than a specific location on a server.

Each file in IPFS is given a unique "content hash," a cryptographic identifier generated from the file's contents. If any part of the file changes, a new hash is created, making each file version easily distinguishable. This process ensures data integrity and enables files to be shared and found by their actual content rather than their physical location. Tools like the Decentralized Encrypted File System (DEFS) build on this feature by maintaining a transparent and comprehensive record of data modifications over time.

In simple terms, let's take an example:

File Creation and Hashing: Suppose you upload a file called hello.txt with the content: "Hello, World!" IPFS generates a unique content hash based on the content inside hello.txt — let's say it's QmXyz123. This hash serves as the file's "address" in IPFS. You can now retrieve this file from IPFS using the hash QmXyz123.

Changing Content Creates a New Hash: Now, let's say you modify the file content to: "Hello, IPFS!" This change will generate a new content hash, say QmAbc456. IPFS stores this version separately; the original file (QmXyz123) remains unchanged. This creates a new "address" for the updated file, making it easy to track different versions by their hashes.

Retrieving the File: When you (or anyone else) want to access hello.txt, you can request it using its hash. IPFS will locate the file on the network by looking for the nodes that store QmXyz123 or QmAbc456 instead of relying on a server address or URL.

Content addressing: In traditional systems, you'd retrieve files through their location (e.g., https://example.com/hello.txt). In IPFS, you fetch files through their content hash, like ipfs://QmXyz123. This approach guarantees that the file retrieved is exactly what you expect, as any change to the content would result in a different hash.

Going Deeper: Content Addressing With Large Files

In IPFS, the content hash remains fixed even if a file is large. IPFS doesn't just create a single hash for the entire file; instead, it breaks large files into smaller chunks, each of which is hashed individually. This method of splitting large files into smaller pieces and hashing each piece allows IPFS to manage large files efficiently without requiring a massive hash.

Here's how it works:

1. Chunking

A large file is added to IPFS and divided into smaller, manageable chunks (usually 256 KB each). Each chunk is individually hashed, creating a unique content hash for that specific chunk.

2. Merkle DAG Structure

After hashing each chunk, IPFS organizes these hashes in a Merkle DAG data structure (Directed Acyclic Graph). In a Merkle DAG, each chunk's hash is linked in a tree-like structure, where "parent" nodes represent combined hashes of "child" chunks. The final result is a single "root hash," uniquely representing the entire file.

3. Single Root Hash

The root hash acts as a "master hash" for the file, pointing to the locations of all chunks that make up the file. So, even though the file might be very large, the root hash remains a fixed length (thanks to IPFS's use of cryptographic hashing algorithms like SHA-256).

4. Efficient, Selective Retrieval

When you request a large file on IPFS, it only downloads the necessary chunks, using the Merkle DAG to locate and verify each chunk by its hash. This structure allows users to efficiently retrieve portions of large files without downloading the entire file if only a portion is required. Also, each chunk's hash is verified as downloaded, ensuring data integrity.

---

IPFS Cluster

Cluster Functionality

The IPFS Cluster component enhances scalability and redundancy by seamlessly pooling storage across a network of servers. By coordinating multiple IPFS nodes, the cluster creates a unified and shared cloud space, allowing all storage across these nodes to work as a single, combined storage layer.

Instead of managing storage on each server individually, all files are handled collectively in the cluster. This design eliminates the need for manual file organization, streamlining storage management directly at the protocol level.

Moreover, the high redundancy built into the cluster ensures exceptional fault tolerance and reliability. Even in server failures, data remains accessible, guaranteeing uninterrupted service. By integrating this advanced feature, Etherland provides a robust and efficient foundation for scalable, secure, and resilient data storage.

Own Infrastructure

Auth and Encryption Server

DEFS integrates decentralized authorization and encryption technologies, leveraging partner DePIN (Decentralized Physical Infrastructure Networks) solutions such as nuco.cloud and VPSAI, which bring specialized capabilities to enhance Etherland's ecosystem and security.

nuco.cloud is a decentralized cloud computing platform that transforms unused computational power into a robust, distributed infrastructure. Its key features relevant to Etherland's ecosystem include:

• Dynamic Scalability: nuco.cloud aggregates unused computing power globally, providing on-demand encryption and data management resources.

• Enhanced Security: Data processed through nuco.cloud benefits from its decentralized architecture, which minimizes single points of failure and reduces vulnerability to cyberattacks.

• Cost Efficiency: Leveraging underutilized resources ensures a cost-effective solution for large-scale encryption tasks, making advanced security accessible and affordable.

• Green Technology: The platform uses otherwise idle computing resources, promoting sustainable IT practices.

Through its integration with nuco.cloud, Etherland achieves enhanced computational capabilities for tasks like secure data encryption, rapid authorization processing, and distributed storage management.

VPSAI is a decentralized platform that combines Virtual Private Servers (VPS) with AI-driven optimization, delivering a powerful, flexible, and secure hosting solution. Key features relevant to Etherland's infrastructure include:

• AI-Enhanced Security: VPSAI uses artificial intelligence to monitor and adapt to emerging security threats in real time, providing proactive defense mechanisms.

• Global Decentralization: By distributing servers across multiple locations, VPSAI ensures high availability and resilience against outages or attacks.

• Encryption Optimization: The platform integrates advanced encryption technologies, ensuring data is protected in transit and at rest.

By leveraging VPSAI, Etherland incorporates cutting-edge AI-powered security and efficient virtual server management, bolstering the reliability of its decentralized storage framework.

By integrating these technologies, Etherland offers a resilient, secure, and high-performance infrastructure that meets the demands of modern decentralized applications.

Storage Infrastructure

The IPFS cluster is designed with a distributed architecture that spans multiple providers across diverse global regions. This approach ensures resilience by deploying storage nodes worldwide, reducing latency, and guaranteeing high availability. Additionally, the system's built-in redundant replication safeguards data by storing multiple copies across nodes, mitigating the risks associated with localized failures and enhancing overall reliability.

Portable Infrastructure

Adaptability

IPFS integrates effortlessly with a client's IT infrastructure, offering tailored storage solutions to meet specific needs. It can be deployed on a client's machines to operate alongside their current systems, enhancing functionality without requiring extensive changes. The protocol's adaptability allows it to seamlessly fit into diverse environments, ensuring a smooth adoption process with minimal disruption to ongoing operations.

Network Mesh

Expanding the number of replicating nodes within an IPFS network strengthens redundancy and significantly boosts storage capacity. With more nodes in the network, fault tolerance is improved, ensuring continuous access to data even in node failures. This scalability also enhances overall storage and retrieval efficiency as additional nodes distribute the load and optimize performance across the network.