Skip to content

Improvements around IPFS timeout #6

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
giliomeejg wants to merge 5 commits into
base: main
Choose a base branch
from
Open

Improvements around IPFS timeout #6

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
8592e0d
func(IPFS): Try retrieving image from own node if we don't get a resp…
giliomeejg May 6, 2022
44678c7
func(Gateway timeout): Add additional check to remove the downloaded …
giliomeejg May 13, 2022
636f894
func(Original image): Remove if the original image looks suspiciously…
giliomeejg May 13, 2022
2ad5ce6
func(Gateway timeout): Improve error handling and cleanup of invalid …
giliomeejg May 13, 2022
c810a70
fix(Filesize): Check that file exists before trying to determine size.
giliomeejg May 13, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 33 additions & 8 deletions app/Http/Controllers/ImageController.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@
class ImageController extends Controller
{

public $maxNrTimesToTryAndProcessAnImage = 3;
private $maxNrTimesToTryAndProcessAnImage = 3; //
private $minimumSizeOfOriginalImage = 180; // The minimum size of an original image. Below this size, remove the original so that a retry occurs.

/**
* Takes a .gif url, width, height and quality as URL parameters.
Expand Down Expand Up @@ -111,8 +112,11 @@ private function getImageIdentifier($url)
*/
private function downloadOriginalImageOrWait($url, $storeLocation, $seconds = 60, $retries = 3)
{
// Use cache to prevent multiple requests to the same image
$cacheName = 'ImageController::downloadOriginalImageOrWait-' . md5($storeLocation);

$this->removeOriginalImageIfItsTooSmall($storeLocation);

if (file_exists(storage_path() . $storeLocation)) {
return;
}
Expand All @@ -123,13 +127,7 @@ private function downloadOriginalImageOrWait($url, $storeLocation, $seconds = 60
// Get a local copy of the image to work with asap and ensure multiple requests to download a local copy don't happen
Cache::put($cacheName, true, 30);
$client = new \GuzzleHttp\Client();
$response = $client->request('GET', $url, ['connect_timeout' => 10, 'sink' => storage_path() . $storeLocation, 'synchronous' => true]);

if ($response->getStatusCode() !== 200) {
// Something went wrong, so remove what was downloaded by $client->request, sink
unlink(storage_path() . $storeLocation);
}

$client->request('GET', $url, ['connect_timeout' => 30, 'sink' => storage_path() . $storeLocation, 'synchronous' => true]);
Cache::forget($cacheName);
} else {
// Wait until we have an original message
Expand All @@ -142,10 +140,37 @@ private function downloadOriginalImageOrWait($url, $storeLocation, $seconds = 60
}
break;
} catch (Exception $e) {
$this->removeOriginalImageIfItsTooSmall($storeLocation);

try {
/**
* If we can't get a response from IPFS, try getting a response from our own node.
*/
$backupUrl = str_replace('https://ipfs.io/ipfs/', 'http://139.59.103.146:8080/ipfs/', $url);
$client = new \GuzzleHttp\Client();
$client->request('GET', $backupUrl, ['connect_timeout' => 30, 'sink' => storage_path() . $storeLocation, 'synchronous' => true]);
} catch (Exception $e) {
$this->removeOriginalImageIfItsTooSmall($storeLocation);
}

Cache::forget($cacheName);
}
}
}

/**
* The way that requests are made, we might end up with an invalid response, e.g. gateway timeout that is stored as a file. In such cases, the file stored is suspiciously small. Remove if needed.
*/
private function removeOriginalImageIfItsTooSmall($storeLocation)
{
if (file_exists(storage_path() . $storeLocation) && filesize(storage_path() . $storeLocation) <= $this->minimumSizeOfOriginalImage) {
// If our original image size is very small, it indicates a potential error, so remove
unlink(storage_path() . $storeLocation);
return true;
}
return false;
}

private function resizeImageFromOriginalIfNeeded(Image $image)
{
// Don't get stuck processing unprocessable images
Expand Down