Skip to content

PGSQL: Tag field too small when importing package from nuget.org #2

@DimeOne

Description

@DimeOne

Describe the bug

When trying to import a package with many tags, the import may fail due to limited field size. Even though the package is hosted at nuget.org which enforces a limit of 4000 characters on tags, the format within the database differs from the format of the nuspec file.

While the nuspec contains a simple text field with blank seperated values, the postgres db contains a json list of all elements.
Considering the that all tags have atleast 2 characters and a blank as seperator would allow for a max of 1334 elements.
The added surplus of the json format can be consired as 4 + ( e * 2 ) - [" as start, "," as seperator and "] as end. So 2 more characters at the start and end, plus 2 more for each element.
This would mean a maximum of 6672 characters, for the sake of clarity rounded up to 7000.

To Reproduce

Steps to reproduce the behavior:

  1. Using the docker container at dime/Baget:7 - Postgresql & Mirror Nuget.org
  2. Trying to import the package by url: https://nuget.buildcache.local/v3/package/nsoftware.secureblackbox/20.0.8300/nsoftware.secureblackbox.20.0.8300.nupkg

Expected behavior

Package is imported and download is possible.

Screenshots

  Microsoft.EntityFrameworkCore.DbUpdateException: An error occurred while saving the entity changes. See the inner exception for details.

   ---> Npgsql.PostgresException (0x80004005): 22001: value too long for type character varying(4000)

     at Npgsql.Internal.NpgsqlConnector.<ReadMessage>g__ReadMessageLong|234_0(NpgsqlConnector connector, Boolean async, DataRowLoadingMode dataRowLoadingMode, Boolean readingNotifications, Boolean isReadingPrependedMessage)

     at Npgsql.NpgsqlDataReader.NextResult(Boolean async, Boolean isConsuming, CancellationToken cancellationToken)

     at Npgsql.NpgsqlDataReader.NextResult(Boolean async, Boolean isConsuming, CancellationToken cancellationToken)

     at Npgsql.NpgsqlCommand.ExecuteReader(CommandBehavior behavior, Boolean async, CancellationToken cancellationToken)

     at Npgsql.NpgsqlCommand.ExecuteReader(CommandBehavior behavior, Boolean async, CancellationToken cancellationToken)

     at Npgsql.NpgsqlCommand.ExecuteDbDataReaderAsync(CommandBehavior behavior, CancellationToken cancellationToken)

     at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReaderAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)

     at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReaderAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)

     at Microsoft.EntityFrameworkCore.Update.ReaderModificationCommandBatch.ExecuteAsync(IRelationalConnection connection, CancellationToken cancellationToken)

    Exception data:

      Severity: ERROR

      SqlState: 22001

      MessageText: value too long for type character varying(4000)

      File: varchar.c

      Line: 638

      Routine: varchar

     --- End of inner exception stack trace ---

     at Microsoft.EntityFrameworkCore.Update.ReaderModificationCommandBatch.ExecuteAsync(IRelationalConnection connection, CancellationToken cancellationToken)

     at Microsoft.EntityFrameworkCore.Update.Internal.BatchExecutor.ExecuteAsync(IEnumerable`1 commandBatches, IRelationalConnection connection, CancellationToken cancellationToken)

     at Microsoft.EntityFrameworkCore.Update.Internal.BatchExecutor.ExecuteAsync(IEnumerable`1 commandBatches, IRelationalConnection connection, CancellationToken cancellationToken)

     at Microsoft.EntityFrameworkCore.Update.Internal.BatchExecutor.ExecuteAsync(IEnumerable`1 commandBatches, IRelationalConnection connection, CancellationToken cancellationToken)

     at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChangesAsync(IList`1 entriesToSave, CancellationToken cancellationToken)

     at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChangesAsync(StateManager stateManager, Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken)

     at Npgsql.EntityFrameworkCore.PostgreSQL.Storage.Internal.NpgsqlExecutionStrategy.ExecuteAsync[TState,TResult](TState state, Func`4 operation, Func`4 verifySucceeded, CancellationToken cancellationToken)

     at Microsoft.EntityFrameworkCore.DbContext.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken)

     at Microsoft.EntityFrameworkCore.DbContext.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken)

     at BaGet.Core.PackageDatabase.AddAsync(Package package, CancellationToken cancellationToken) in /src/BaGet.Core/PackageDatabase.cs:line 35

     at BaGet.Core.PackageIndexingService.IndexAsync(Stream packageStream, CancellationToken cancellationToken) in /src/BaGet.Core/Indexing/PackageIndexingService.cs:line 163

     at BaGet.Core.PackageService.MirrorAsync(String id, NuGetVersion version, CancellationToken cancellationToken) in /src/BaGet.Core/PackageService.cs:line 147

Additional context

Add any other context about the problem here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions