Error - Premature end of stream (Kartoffels)

[Calciphel]

When attempting to download any of Kartoffels Texture collections I receive the error below. On attempting to disable the Heliosphere plugin and download as a pmp, the process never completes and does not start the actual download. Anyone have any ideas? Is this something on my end or on the platform's end?

Edit: For clarification, I am able to download other items. This only seems to happen with Kartoffels collections.

Error type: System.IO.EndOfStreamException
   Message: Premature end of stream
   HResult: 0x80070026
   at ZstdSharp.DecompressionStream.ReadAsync(Memory`1 buffer, CancellationToken cancellationToken)
   at Heliosphere.DownloadTask.DownloadBatchedFile(IDownloadTask_GetVersion_NeededFiles neededFiles, String filesPath, Uri uri, RangeHeaderValue rangeHeader, IReadOnlyList`1 chunks, IReadOnlyDictionary`2 batchedFiles, StateCounter counter) in /_/DownloadTask.cs:line 804
   at Heliosphere.DownloadTask.DownloadBatchedFile(IDownloadTask_GetVersion_NeededFiles neededFiles, String filesPath, Uri uri, RangeHeaderValue rangeHeader, IReadOnlyList`1 chunks, IReadOnlyDictionary`2 batchedFiles, StateCounter counter) in /_/DownloadTask.cs:line 828
   at Heliosphere.DownloadTask.DownloadBatchedFile(IDownloadTask_GetVersion_NeededFiles neededFiles, String filesPath, Uri uri, RangeHeaderValue rangeHeader, IReadOnlyList`1 chunks, IReadOnlyDictionary`2 batchedFiles, StateCounter counter) in /_/DownloadTask.cs:line 828
   at Heliosphere.DownloadTask.DownloadBatchedFile(IDownloadTask_GetVersion_NeededFiles neededFiles, String filesPath, Uri uri, RangeHeaderValue rangeHeader, IReadOnlyList`1 chunks, IReadOnlyDictionary`2 batchedFiles, StateCounter counter) in /_/DownloadTask.cs:line 828
   at Heliosphere.DownloadTask.DownloadBatchedFile(IDownloadTask_GetVersion_NeededFiles neededFiles, String filesPath, Uri uri, RangeHeaderValue rangeHeader, IReadOnlyList`1 chunks, IReadOnlyDictionary`2 batchedFiles, StateCounter counter) in /_/DownloadTask.cs:line 778
   at Heliosphere.DownloadTask.DownloadBatchedFile(IDownloadTask_GetVersion_NeededFiles neededFiles, String filesPath, Uri uri, RangeHeaderValue rangeHeader, IReadOnlyList`1 chunks, IReadOnlyDictionary`2 batchedFiles, StateCounter counter) in /_/DownloadTask.cs:line 778
   at Heliosphere.DownloadTask.<>c__DisplayClass130_1.<<DownloadBatchedFiles>b__10>d.MoveNext() in /_/DownloadTask.cs:line 663
--- End of stack trace from previous location ---
   at Polly.ResiliencePipeline.<>c.<<ExecuteAsync>b__3_0>d.MoveNext()
--- End of stack trace from previous location ---
   at Polly.ResiliencePipeline.ExecuteAsync(Func`2 callback, CancellationToken cancellationToken)
   at Heliosphere.DownloadTask.<>c__DisplayClass130_0.<<DownloadBatchedFiles>b__2>d.MoveNext() in /_/DownloadTask.cs:line 657
--- End of stack trace from previous location ---
   at System.Threading.Tasks.Parallel.<>c__53`1.<<ForEachAsync>b__53_0>d.MoveNext()
--- End of stack trace from previous location ---
   at Heliosphere.DownloadTask.DownloadFiles(IDownloadTask_GetVersion info) in /_/DownloadTask.cs:line 505
   at Heliosphere.DownloadTask.Run() in /_/DownloadTask.cs:line 167

 

[ShiningKestral]

Hi, when I attempt to install certain mods (seems to happen on mods with 800+ files) I get EndOfStreamException errors. I cannot install them with the plugin nor can I download them as .pmp files.

Error type: System.IO.EndOfStreamException
   Message: Premature end of stream
   HResult: 0x80070026
   at ZstdSharp.DecompressionStream.ReadAsync(Memory`1 buffer, CancellationToken cancellationToken)
   at Heliosphere.DownloadTask.DownloadBatchedFile(IDownloadTask_GetVersion_NeededFiles neededFiles, String filesPath, Uri uri, RangeHeaderValue rangeHeader, IReadOnlyList`1 chunks, IReadOnlyDictionary`2 batchedFiles, StateCounter counter) in /_/DownloadTask.cs:line 804
   at Heliosphere.DownloadTask.DownloadBatchedFile(IDownloadTask_GetVersion_NeededFiles neededFiles, String filesPath, Uri uri, RangeHeaderValue rangeHeader, IReadOnlyList`1 chunks, IReadOnlyDictionary`2 batchedFiles, StateCounter counter) in /_/DownloadTask.cs:line 828
   at Heliosphere.DownloadTask.DownloadBatchedFile(IDownloadTask_GetVersion_NeededFiles neededFiles, String filesPath, Uri uri, RangeHeaderValue rangeHeader, IReadOnlyList`1 chunks, IReadOnlyDictionary`2 batchedFiles, StateCounter counter) in /_/DownloadTask.cs:line 828
   at Heliosphere.DownloadTask.DownloadBatchedFile(IDownloadTask_GetVersion_NeededFiles neededFiles, String filesPath, Uri uri, RangeHeaderValue rangeHeader, IReadOnlyList`1 chunks, IReadOnlyDictionary`2 batchedFiles, StateCounter counter) in /_/DownloadTask.cs:line 828
   at Heliosphere.DownloadTask.DownloadBatchedFile(IDownloadTask_GetVersion_NeededFiles neededFiles, String filesPath, Uri uri, RangeHeaderValue rangeHeader, IReadOnlyList`1 chunks, IReadOnlyDictionary`2 batchedFiles, StateCounter counter) in /_/DownloadTask.cs:line 778
   at Heliosphere.DownloadTask.DownloadBatchedFile(IDownloadTask_GetVersion_NeededFiles neededFiles, String filesPath, Uri uri, RangeHeaderValue rangeHeader, IReadOnlyList`1 chunks, IReadOnlyDictionary`2 batchedFiles, StateCounter counter) in /_/DownloadTask.cs:line 778
   at Heliosphere.DownloadTask.<>c__DisplayClass130_1.<<DownloadBatchedFiles>b__10>d.MoveNext() in /_/DownloadTask.cs:line 663
--- End of stack trace from previous location ---
   at Polly.ResiliencePipeline.<>c.<<ExecuteAsync>b__3_0>d.MoveNext()
--- End of stack trace from previous location ---
   at Polly.ResiliencePipeline.ExecuteAsync(Func`2 callback, CancellationToken cancellationToken)
   at Heliosphere.DownloadTask.<>c__DisplayClass130_0.<<DownloadBatchedFiles>b__2>d.MoveNext() in /_/DownloadTask.cs:line 657
--- End of stack trace from previous location ---
   at System.Threading.Tasks.Parallel.<>c__53`1.<<ForEachAsync>b__53_0>d.MoveNext()
--- End of stack trace from previous location ---
   at Heliosphere.DownloadTask.DownloadFiles(IDownloadTask_GetVersion info) in /_/DownloadTask.cs:line 505
   at Heliosphere.DownloadTask.Run() in /_/DownloadTask.cs:line 167

 

[anna]

Yeah, I can reproduce this. Looking into it.

Edit: the file is not corrupt, so it seems like a bug in the download logic for this type of batched file. I'll need to investigate more.

Edit 2: I think this is a big Cloudflare issue. If so, there's nothing I can do right now. More investigation is required still.

Edit 3: Definitely a Cloudflare issue and I'm looking into reporting it/getting help. This may prove to be a very big headache.

 

[ShiningKestral]

Thank you for looking into this.

If it helps, I can list the mods that have I also tried that failed to install/download, meaning the issue is not limited to Kartoffels:

Hair Defined 2 - Ultra (UHD 4x Variant, 7.X Patch Updates works)
Otopop Mani-Pedi
Otopop Mega Vanilla Conversion Pack (All versions)

 

[UVFrost]

For what it may be worth, I'm having the same error with Everyone Bites.

 

[anna]

Yeah, it's going to be mods with (if I recall correctly) more than 500 files. Cloudflare is returning incorrect data for their files.

To be clear: the files are stored correctly. Not a corruption issue. It's an issue with how they're being requested and returned from storage.

 

[anna]

As an update, last night I was able to talk to a Cloudflare employee who was able to find an issue on their side. No ETA on a fix, but hopefully they've found the right bug and are able to fix it soon.

 

[richerich]

Yeah, i was just about to reply in another thread about this... I was able to download a older version of EB (v.4.1.6 specifically), but any version above that and the update fails, regardless if i use the plugin or website directly... even if i attempt to download it manually, it still failed.
Good to see it's known and at least being looked into.

 

[CuddleBringer]

Yeah, i was just about to reply in another thread about this... I was able to download a older version of EB (v.4.1.6 specifically), but any version above that and the update fails, regardless if i use the plugin or website directly... even if i attempt to download it manually, it still failed.
Good to see it's known and at least being looked into.

i've been having issues getting EB to update. wasn't aware that it was a cloudflare issue, and just assumed it was a client side issue. ended up uninstalling the version of EB i had (4.2.5), and currently am unable to get any version of it to install.

 

[anna]

Locking this thread. It's not helpful to have people repeat the issue several times. I'll post when I see the problem get fixed.

 

[anna]

I have pushed an update to the plugin (4.2.1) that should work around these issues. Downloads will be slightly slower and waste bandwidth for the affected mods, but they will successfully download at least. The underlying issue is not yet fixed.