feat(shwap/blob): add blob to shwap by vgonkivs · Pull Request #4765 · celestiaorg/celestia-node

vgonkivs · 2026-01-22T16:45:46Z

The PR adds a support of Blob type request and response for the shrex/protocol. Please note that the PR is breaking as it breaks the proof verification logic: instead of proving the whole namespace within the Row, blob Proof now tied to the specific blob.

renaynay

Some questions:

what is the point of GetBlobs to be implemented over blob over shwap -- why wouldn't it just route to namespace data over shwap? Is this just an optimisation not to send padding if it exists? If so, then what's the point of shares by namespace over shwap (besides namespace non-inclusion)?
similarly, GetBlobs() feels to me like it should be a message at shwap protocol level for fetching multiple blobs by commitment rather than just all -- right now, we can only fetch 1 blob by commitment over shwap or fetching ALL blobs in that namespace. Is there a case that we would need that? fetching multiple blobs by commitment from the same namespace? And not to bikeshed, but similarly, would there be a need to fetch blobs from different namespaces but at the same height such that it would be more efficient to batch it in one request?
Can we DRY between Blob()/Blobs() methods between rsmt2d.go and ods.go
Why do we need two methods on accessor? Blob + Blobs? Can we not just have 1 (Blobs) and pass in 1 or multiple there? Right now, this is probably b/c we have the binary 1 blob or ALL blobs.
feels like missing lots of unit tests

renaynay · 2026-02-09T13:34:29Z

share/shwap/getters/cascade.go

+	ctx, span := tracer.Start(
+		ctx,
+		"cascade/get-blob",
+		trace.WithAttributes(


add commitment here

renaynay · 2026-02-09T13:35:12Z

share/eds/rsmt2d.go

+	return shwap.BlobFromShares(extendedRowShares, namespace, commitment, odsSize)
+}
+
+func (eds *Rsmt2D) Blobs(ctx context.Context, namespace libshare.Namespace) ([]*shwap.Blob, error) {


where are unit tests for these methods

renaynay · 2026-02-09T13:36:34Z

share/shwap/p2p/shrex/shrex_getter/shrex.go

+	commitment []byte,
+) (*shwap.Blob, error) {
+	var err error
+	ctx, span := tracer.Start(ctx, "shrex/get-blob", trace.WithAttributes(


we need to pull span from the trace that's passed through cascade getter rather than starting new one

This is the problem of all methods in shrex. Probably, I missed this part during shrex refactoring. I'll prepare a PR to fix all at once.

renaynay · 2026-02-09T13:37:02Z

share/shwap/p2p/shrex/shrex_getter/shrex.go

+	ns libshare.Namespace,
+) ([]*shwap.Blob, error) {
+	var err error
+	ctx, span := tracer.Start(ctx, "shrex/get-blobs", trace.WithAttributes(


renaynay · 2026-02-09T13:39:09Z

share/shwap/p2p/shrex/shrex_getter/shrex.go

+			return errors.New("nil response")
+		}
+		for _, blob := range response {
+			err = blob.VerifyInclusion(header.DAH.RowRoots)


why are we only calling VerifyInclusion here instead of top-level Verify?

We don't know the commitments during GetBlobs. So we are interested only in inclusion.

renaynay · 2026-02-09T14:22:40Z

share/shwap/blob.go

+	}, nil
+}
+
+func BlobsFromShares(


Was all of this code moved from the blob pkg down here?

renaynay · 2026-02-09T14:33:05Z

share/shwap/blob_id.go

+
+// BlobIDSize defines the total size of a BlobID in bytes, combining the
+// size of a EdsID, the size of a Namespace and the commitment size
+const BlobIDSize = EdsIDSize + libshare.NamespaceSize + commitmentSize


is there any test coverage for this file?

renaynay · 2026-02-09T14:33:53Z

share/shwap/blob_id.go

+func (blbID *BlobID) ResponseReader(ctx context.Context, acc Accessor) (io.Reader, error) {
+	buf := &bytes.Buffer{}
+
+	if !bytes.Equal(blbID.Commitment, emptyCommitment) {


Switch case on here to make more readable?

renaynay · 2026-02-09T14:35:15Z

share/shwap/range_namespace_data.go

+
+	// Make a copy of the outer slice to avoid modifying the caller's slice
+	// when we reassign row elements below.
+	shares := make([][]libshare.Share, len(extendedRowShares))


can we extract into separate PR as this is a big feature PR and would rather not bundle other stuff into

This change is necessary and imo should be a part of this PR. Previously, a single range was requested which allows us to modify the outer slice. In case of blob's retrieval it will lead to an error with indexing, out of bounds errors etc.

renaynay · 2026-02-09T14:36:05Z

store/file/ods.go

 	return shwap.RangeNamespaceDataFromShares(shares, fromCoords, toCoords)
 }

+func (o *ODS) Blob(ctx context.Context, namespace libshare.Namespace, commitment []byte) (*shwap.Blob, error) {


where are units?

already working on them

vgonkivs · 2026-02-10T10:24:32Z

what is the point of GetBlobs to be implemented over blob over shwap -- why wouldn't it just route to namespace data over shwap? Is this just an optimisation not to send padding if it exists? If so, then what's the point of shares by namespace over shwap (besides namespace non-inclusion)?

The point is to have a proof, associated with blob and not for the whole namespace in the row.

similarly, GetBlobs() feels to me like it should be a message at shwap protocol level for fetching multiple blobs by commitment rather than just all -- right now.

You can request multiple blobs by commitment from different peers(as GetSamples) and make requests parallel. We can add this if it will be requested.
The point of give all blobs from this namespace is to support blob subscription.

And not to bikeshed, but similarly, would there be a need to fetch blobs from different namespaces but at the same height such that it would be more efficient to batch it in one request?

We are not requesting samples as batches from the security standpoint. Maybe we should follow this approach with blobs as well?

Can we DRY between Blob()/Blobs() methods between rsmt2d.go and ods.go

Yes

Why do we need two methods on accessor? Blob + Blobs? Can we not just have 1 (Blobs) and pass in 1 or multiple there? Right now, this is probably b/c we have the binary 1 blob or ALL blobs.

Right. Thank you.

feels like missing lots of unit tests

Already working on them.

walldiss · 2026-02-16T12:29:48Z

nodebuilder/da/service.go

 	if err != nil {
-		if strings.Contains(err.Error(), blob.ErrBlobNotFound.Error()) {
+		if strings.Contains(err.Error(), shwap.ErrBlobNotFound.Error()) {
 			return nil, nil //nolint:nilnil
 		}


hmm, why don't we return non-inclusion proof?

We havent returned it previously

walldiss · 2026-02-16T12:33:37Z

share/shwap/getter.go

+	// GetBlobs retrieves all blobs belonging to the given namespace from the
+	// Extended Data Square (EDS) referenced by the given header.
+	GetBlobs(_ context.Context, _ *header.ExtendedHeader, _ libshare.Namespace) ([]*Blob, error)


Why you added this method? It seems to duplicate blobs parsing from blob service but leaking into reader level for some reason.

walldiss · 2026-02-16T12:36:37Z

share/eds/accessor.go

+
+	// Blobs retrieves blobs belonging to the given namespace from the data square.
+	// When commitments are provided, only blobs matching those commitments are returned.
+	// When no commitments are provided, all blobs in the namespace are returned.
+	Blobs(context.Context, libshare.Namespace, ...[]byte) ([]*shwap.Blob, error)


It is a bit confusing why some APIs and interfaces are requesting single blobs while others are requesting multiple. We used to have ability to request only one blob by commitment. Are there some fundamentl changes required to make this PR? Imo ability to request multiple blobs by commitment at the same time is out of scope of this change and should be discussed / implemented separetly.

Top API level contains 2 independent methods(GetBlob/GetBlobs), while accessor has only one, since the logic is pretty the same and @renaynay suggested to have only one method.

walldiss · 2026-02-16T12:45:54Z

share/eds/rsmt2d.go

+func (eds *Rsmt2D) extendedRowShares(ctx context.Context) ([][]libshare.Share, int, error) {
+	size, err := eds.Size(ctx)
+	if err != nil {
+		return nil, 0, err
+	}
+
+	odsSize := size / 2
+	shares := make([][]libshare.Share, odsSize)
+	for index := 0; index < odsSize; index++ {
+		rawShare := eds.Row(uint(index))
+		sh, err := libshare.FromBytes(rawShare)
+		if err != nil {
+			return nil, 0, err
+		}
+		shares[index] = sh
+	}
+	return shares, odsSize, nil
+}


It is duplication. You can get extended row from just Row method.

github-actions bot added the kind:break! Attached to breaking PRs label Jan 22, 2026

vgonkivs self-assigned this Jan 23, 2026

vgonkivs force-pushed the add_blob_over_shrex branch 6 times, most recently from 05f41a8 to f6bf9f1 Compare January 25, 2026 08:55

vgonkivs marked this pull request as ready for review January 25, 2026 09:05

vgonkivs requested review from Wondertan, renaynay and walldiss as code owners January 25, 2026 09:06

vgonkivs changed the title ~~add blob to shwap~~ feat!(shwap/blob): add blob to shwap Jan 26, 2026

vgonkivs added the kind:feat Attached to feature PRs label Jan 26, 2026

vgonkivs changed the title ~~feat!(shwap/blob): add blob to shwap~~ feat(shwap/blob): add blob to shwap Feb 9, 2026

vgonkivs removed the kind:break! Attached to breaking PRs label Feb 9, 2026

renaynay reviewed Feb 9, 2026

View reviewed changes

vgonkivs added 5 commits February 16, 2026 13:26

add blob to shwap

deee3b4

fix lint

e21b224

add some fixes

d31839e

revert changes in blob service

6b50217

apply suggestions + fix findings

3c825d6

vgonkivs force-pushed the add_blob_over_shrex branch from 92b984c to 3c825d6 Compare February 16, 2026 11:36

vgonkivs requested a review from mcrakhman as a code owner February 16, 2026 11:36

add commitment to span

0af1226

walldiss reviewed Feb 16, 2026

View reviewed changes

Conversation

vgonkivs commented Jan 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

renaynay left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vgonkivs Feb 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vgonkivs commented Feb 10, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vgonkivs Feb 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

vgonkivs commented Jan 22, 2026 •

edited

Loading

vgonkivs Feb 16, 2026 •

edited

Loading

vgonkivs Feb 16, 2026 •

edited

Loading