Proposal: Allow monthly file sharding as alternative to daily

[melvincarvalho]

Summary

The current spec mandates daily files (YYYY/MM/DD/chat.ttl). This proposal suggests allowing monthly files (YYYY/MM.ttl) as an alternative for moderate-activity chat rooms.

Problem with Daily-Only

For rooms with low-to-moderate activity (~10-100 messages/day):

Metric	Daily	Monthly
Files per year	365	12
HTTP requests to load 1 month	~30	1
Avg file size	10-50KB	300KB-1MB
Directory listings needed	Many	Few

Daily sharding creates:

• Many small files (overhead)
• Many HTTP requests for historical browsing
• Complex discovery (need to enumerate days)

Proposal

Allow implementations to choose sharding granularity:

<#chat> a meeting:LongChat ;
    :shardingPeriod :monthly .  # or :daily (default)

Monthly structure:

/chat/
  index.ttl
  2024/
    01.ttl
    02.ttl

Daily structure (current, remains default):

/chat/
  index.ttl
  2024/
    01/
      15/chat.ttl
      16/chat.ttl

Discovery

Readers would check index.ttl for sharding type, then:

• Daily: enumerate YYYY/MM/DD/
• Monthly: enumerate YYYY/MM.ttl

Or better: index file lists all shard URIs explicitly.

Use Cases

Room Type	Recommended
High-volume (100+ msgs/day)	Daily
Moderate (10-100 msgs/day)	Monthly
Low-volume (<10 msgs/day)	Monthly or even yearly

Backwards Compatibility

• Daily remains the default
• Existing daily chats continue working
• Monthly is opt-in for new chats
• Readers must support both (follow links from index)

References

• Discussion: Scalability: Handle long chat histories sustainably solid-chat/app#7
• Current spec: https://solid.github.io/chat/

Would love feedback on whether this flexibility would be acceptable while maintaining interoperability.

[melvincarvalho]

Additional Proposal: Linked List for Day Discovery

Regardless of daily vs monthly sharding, there's a discovery problem: how does a client find previous day files without:

• Probing with 404s (inefficient)
• Directory listing (not always supported)
• Maintaining an index file (extra complexity)

Proposal: Each day file links to its predecessor

# 2025/01/09/chat.ttl
@prefix : <#> .
@prefix meeting: <http://www.w3.org/ns/pim/meeting#> .

:this a meeting:LongChat ;
    :previousChatDay <../../01/08/chat.ttl> .

# messages...

How it works

[Jan 9] ──prev──▶ [Jan 8] ──prev──▶ [Jan 5] ──prev──▶ [Jan 3] ──prev──▶ (none)
                   (empty days are skipped automatically)

Benefits

Benefit	Explanation
No index file	Each file is self-describing
No 404 probing	Skip empty days automatically
No directory listing	Works on any Solid server
Lazy loading	Only fetch when user scrolls up
Decentralized	Like git commits - each points to parent

Client implementation

// Walking backwards
async function loadOlderMessages(currentDayUri) {
  const store = await fetchAndParse(currentDayUri)
  const prevLink = store.any(null, PREV_CHAT_DAY)?.value
  if (prevLink) {
    const messages = await loadMessages(prevLink)
    return { messages, prevUri: prevLink }
  }
  return null  // End of history
}

On new day creation

When creating the first message of a new day:

const previousDayUri = await findMostRecentExistingDay()

const turtle = `
:this a meeting:LongChat ;
    :previousChatDay <${previousDayUri}> .
`
// PUT to new day file

Combined with monthly sharding

This approach works with both daily and monthly files:

• Daily: 2025/01/09/chat.ttl → 2025/01/08/chat.ttl
• Monthly: 2025/01.ttl → 2024/12.ttl

The linked list pattern solves discovery independent of sharding granularity.