Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions Architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -163,7 +163,7 @@ flowchart LR
Agg --> Client[Client App]
```

**Use Case**: Social feeds, search, discovery (e.g., [[Explore/PubkyApp/Backend/PubkyNexus|Pubky Nexus]])
**Use Case**: Social feeds, search, discovery (e.g., [[Explore/PubkyApps/IndexingAndAggregation/PubkyNexus|Pubky Nexus]])

#### 3. Custom Backend

Expand Down Expand Up @@ -232,7 +232,7 @@ sequenceDiagram

### Pubky Nexus

**[[Explore/PubkyApp/Backend/PubkyNexus|Pubky Nexus]]** provides:
**[[Explore/PubkyApps/IndexingAndAggregation/PubkyNexus|Pubky Nexus]]** provides:
- Real-time aggregation
- Social graph indexing
- Search and discovery
Expand Down
4 changes: 2 additions & 2 deletions Contributing.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@ Help improve this knowledge base:
Contribute to Pubky projects:
- **[[Explore/PubkyCore/Introduction|Pubky Core]]**: Protocol and Homeserver
- **[[Explore/Technologies/PubkyRing|Pubky Ring]]**: Mobile key manager
- **[[Explore/PubkyApp/Introduction|Pubky App]]**: Social application ([pubky.app](https://pubky.app))
- **[[Explore/PubkyApp/Backend/PubkyNexus|Pubky Nexus]]**: Indexing service
- **[[Explore/PubkyApps/Introduction|Pubky Apps]]**: Social application ([pubky.app](https://pubky.app))
- **[[Explore/PubkyApps/IndexingAndAggregation/PubkyNexus|Pubky Nexus]]**: Indexing service
- **[[Explore/Technologies/PubkyCLI|Pubky CLI]]**: Command-line tool
- **[[PKDNS]]**, **[[Homegate]]**, **[[PubkyDocker|Pubky Docker]]**: Infrastructure tools

Expand Down
8 changes: 0 additions & 8 deletions Explore/PubkyApp/AppArchitectures/index.md

This file was deleted.

20 changes: 0 additions & 20 deletions Explore/PubkyApp/Backend/Introduction.md

This file was deleted.

6 changes: 0 additions & 6 deletions Explore/PubkyApp/Client/Features/Layouts.md

This file was deleted.

50 changes: 0 additions & 50 deletions Explore/PubkyApp/Introduction.md

This file was deleted.

8 changes: 0 additions & 8 deletions Explore/PubkyApp/index.md

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
title: "Introduction"
aliases:
- "../../Pubky-App/App-Architectures/1.-Introduction"
- "Explore/PubkyApp/AppArchitectures/1.Introduction"
---

Leveraging the [[Explore/PubkyCore/Introduction|Pubky Core]] protocol as the foundational layer, we can envision a diverse array of applications with distinct functionalities. While each app can implement its own unique design patterns and user interfaces, they all share a common underlying architecture: interacting with the distributed data stores, colloquially referred to as [[Homeserver|Homeservers]], through standardized read and write operations.
Expand All @@ -10,6 +11,6 @@ The [[Explore/PubkyCore/Introduction|Pubky Core protocol]] enables a decentraliz

Below, we present several high-level designs that showcase the versatility of this architecture, from simple client (browser) apps that interact directly with one or several [[Homeserver|Homeservers]] to more complex applications with custom aggregators and backends capable of powerful inference:

- [[2.ClientHomeserver|Client-Homeserver]]
- [[2.ClientHomeserver|Homeserver-only]]
- [[4.CustomBackend|Custom backend]]
- [[3.GlobalAggregators|Global aggregators]]
Original file line number Diff line number Diff line change
@@ -1,19 +1,20 @@
---
title: "Client - Homeserver"
title: "Homeserver-only app"
aliases:
- "../../Pubky-App/App-Architectures/2.-Client---Homeserver"
- "Explore/PubkyApp/AppArchitectures/2.ClientHomeserver"
---

![Simple client-homeserver architecture diagram showing direct connection between client application and a single Homeserver for data storage and retrieval](Explore/images/client-homeserver.png)
![Homeserver-only architecture diagram showing an application reading and writing directly to a Homeserver, with PKARR relay for discovery](Explore/images/client-homeserver.png)

In this architecture, we implement a direct communication model between the client application and the Homeserver. This approach minimizes latency and reduces system complexity by establishing a direct data flow pathway.
In this architecture, the application communicates directly with the Homeserver. This approach minimizes latency and reduces system complexity by establishing a direct data flow pathway.

This design pattern is particularly well-suited for applications with straightforward functionality, especially those that don't require real-time interaction or data normalization. This architectural approach demonstrates optimal performance in use cases characterized by intermittent data operations, where asynchronous read/write cycles are adequate for maintaining data consistency and fulfilling application requirements.

To illustrate the practical applications of this architectural paradigm, consider the following implementation scenarios:

1. Bookmark Management System: A client application designed to store and retrieve user bookmarks directly from the Homeserver.
2. File Synchronization Utility: Similar to the open-source [Syncthing](https://syncthing.net/) project, this type of application would facilitate direct file synchronization between the client and the Homeserver.
1. Bookmark Management System: An application designed to store and retrieve user bookmarks directly from the Homeserver.
2. File Synchronization Utility: Similar to the open-source [Syncthing](https://syncthing.net/) project, this type of application would facilitate direct file synchronization with the Homeserver.
3. Text Snippet Repository: A lightweight application for creating, storing, and retrieving short text fragments or code snippets as [pastebin](https://pastebin.com/)

These implementations leverage the Pubky Core protocol to establish secure, efficient, and direct data exchange channels between the client and the Homeserver, while the user remains with ownership of their data.
These implementations leverage the Pubky Core protocol to establish secure, efficient, and direct data exchange channels with the Homeserver, while the user remains with ownership of their data.
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,16 @@
title: "Global Aggregators"
aliases:
- "../../Pubky-App/App-Architectures/3.-Global-Aggregators"
- "Explore/PubkyApp/AppArchitectures/3.GlobalAggregators"
---

![Global aggregator architecture diagram showing multiple Homeservers feeding data into a central aggregator service that provides unified API to clients](Explore/images/global_aggregator.png)
![Global aggregator architecture diagram showing multiple Homeservers feeding data into a central aggregator service that provides a unified API to Pubky apps](Explore/images/global_aggregator.png)

This architectural pattern implements a distributed system model centered around a global aggregation layer, eliminating the need to fetch data from a multitude (maybe thousands!) of Homeservers by the client. The core component of this design is a centralized global aggregator that interfaces with multiple Homeservers, consuming events from each in a unified manner.
This architectural pattern implements a distributed system model centered around a global aggregation layer, eliminating the need to fetch data from a multitude (maybe thousands!) of Homeservers by the Pubky app. The core component of this design is a centralized global aggregator that interfaces with multiple Homeservers, consuming events from each in a unified manner.

Key features of this architecture include:

1. **Centralized Event Processing:** The global aggregator serves as a single point of convergence for event streams originating from disparate Homeservers across the network.
2. **Policy-Driven Filtering:** The aggregators can optionally implement a configurable set of policies and filtering rules, allowing for dynamic event processing based on predefined criteria.
3. **Client Flexibility and Aggregator Choice:** Clients consume data from the global aggregator stream. However, if a client finds the enforced rules of one aggregator unsuitable, it retains the flexibility to switch to an alternative global aggregator that better aligns with its requirements or selectively look for the Homeservers itself.
4. **Scalable Event Distribution:** By centralizing the aggregation process, this architecture facilitates efficient event distribution to multiple clients, potentially reducing redundant processing and network overhead.
3. **Pubky App Flexibility and Aggregator Choice:** Pubky apps consume data from the global aggregator stream. However, if a Pubky app finds the enforced rules of one aggregator unsuitable, it retains the flexibility to switch to an alternative global aggregator that better aligns with its requirements or selectively look for the Homeservers itself.
4. **Scalable Event Distribution:** By centralizing the aggregation process, this architecture facilitates efficient event distribution to multiple Pubky apps, potentially reducing redundant processing and network overhead.
Original file line number Diff line number Diff line change
Expand Up @@ -2,31 +2,32 @@
title: "Custom Backend"
aliases:
- "../../Pubky-App/App-Architectures/4.-Custom-Backend"
- "Explore/PubkyApp/AppArchitectures/4.CustomBackend"
---

![Complex custom backend architecture diagram showing Homeservers, custom aggregation logic, inference engines, and specialized API services](Explore/images/custom_backend.png)
![Custom backend architecture diagram showing Homeservers streaming data to a backend layer that serves Pubky apps](Explore/images/custom_backend.png)

This architectural design introduces a more sophisticated data flow model, incorporating an intermediary backend layer between the client application and the Homeserver. This backend functions as a middleware, enhancing the system's flexibility and data processing capabilities.
This architectural design introduces a more sophisticated data flow model, incorporating an intermediary backend layer between the Pubky app and the Homeserver. This backend functions as a middleware, enhancing the system's flexibility and data processing capabilities.

The backend can be potentially comprised with many components. These components will depend on the client app needs, but these are the main ones
The backend can be potentially comprised with many components. These components will depend on the Pubky app needs, but these are the main ones

1. __Indexer__: Responsible for data normalization, ensuring consistent data structures and optimizing query performance.
2. __Aggregator__: Implements event filtering logic, allowing for selective data propagation based on predefined criteria.


This architecture supports two distinct data consumption patterns:

a) For scenarios requiring both data normalization and event filtering, the client interacts with the backend layer, as an endpoint. The aggregator processes the event stream from the Homeservers, applying filtering rules before passing the data to the indexer for normalization.
a) For scenarios requiring both data normalization and event filtering, the Pubky app interacts with the backend layer, as an endpoint. The aggregator processes the event stream from the Homeservers, applying filtering rules before passing the data to the indexer for normalization.

b) In cases where only data normalization is necessary, the backend can bypass or not implement the aggregator, consuming events directly from the Homeserver via the indexer.

This modular approach allows for fine-grained control over data processing, enabling efficient resource utilization and optimized client-side performance based on specific application requirements.
This modular approach allows for fine-grained control over data processing, enabling efficient resource utilization and optimized Pubky app performance based on specific application requirements.

## Pubky Nexus: Production Implementation

[[PubkyNexus|Pubky Nexus]] is the production-grade implementation of this custom backend architecture. It combines sophisticated aggregation, normalization, indexation, filtering (compliance), and powerful Social Semantic Graph inference capabilities.

Nexus powers the [[Explore/PubkyApp/Introduction|Pubky App]] social features with:
Nexus powers [[Explore/PubkyApps/Introduction|pubky.app]]'s social features with:
- Real-time event aggregation from multiple Homeservers
- High-performance graph database (Neo4j) for relationship queries
- Redis caching layer for sub-millisecond response times
Expand Down
10 changes: 10 additions & 0 deletions Explore/PubkyApps/AppArchitectures/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
title: "App Architectures"
aliases:
- "Explore/PubkyApp/AppArchitectures/index"
---

- [[Explore/PubkyApps/AppArchitectures/1.Introduction|Introduction]]
- [[Explore/PubkyApps/AppArchitectures/2.ClientHomeserver|Homeserver-only]]
- [[Explore/PubkyApps/AppArchitectures/3.GlobalAggregators|Global Aggregators]]
- [[Explore/PubkyApps/AppArchitectures/4.CustomBackend|Custom Backend]]
Loading