New Logging Output

3 posts / 0 new
Last post
puravida's picture
puravida
Jedi Warrior
Offline
Joined: 09/01/2007
Visit puravida's Website

Hello,

We have stumbled across a type of error where our cached image is missing from our storage facility. This situation is automatically corrected when users access our primary delivery script (xino) but was not handled in the rare cases that users were accessing the advanced API image delivery script (xian) directly. We do not recommend bypassing the primary delivery script (xino) in any case, but some integrations store the image path provided in the XML response and use that for follow-up requests. That is not "Best Practice" but we decided to implement some checks to detect missing images in this case and automatically refresh the request for free.

This new logging output and refresh feature will not affect very many users, but a rare few may notice "Image Retrieval Error" in the logs while the system refreshes any missing images. Once the bulk of the missing images are fixed, we do not anticipate the missing images to be an on-going issue. However, this new logging will help us to see if it is an on-going issue and we will investigate accordingly.

Sincerely,

Brandon

puravida's picture
puravida
Jedi Warrior
Offline
Joined: 09/01/2007
Visit puravida's Website

It has become clear that there is another issue in play here; not just missing images.

So, for missing images, we are logging "Cached Image Missing" for those from now on, and will be using "Image Retrieval Error" to indicate a failed connection or lack of response from our storage provider.

puravida's picture
puravida
Jedi Warrior
Offline
Joined: 09/01/2007
Visit puravida's Website

[GEEK-SPEAK]
After a couple of weeks of in-depth troubleshooting on our servers and working with our storage provider's engineers as well, we believe that we have finally gotten to the bottom of the "Image Retrieval Error" issue (which has been increasing in frequency).

It took a lot of debugging and error reporting, because in the overall scheme of things, this issue was only occurring about 1:1,000,000 requests, which made it extremely unlikely to manually reproduce the error condition. We also had no way to "catch" this error LIVE, while still gracefully failing on the client side, since every means to gather error details resulted in NO data at all. Short of throwing errors in the LIVE service, I finally stumbled across the thought that it might be rate-limit related and was able to reproduce this error condition manually. It was then that I learned that these headaches have been a result of intermittent rate-limit DNS failures when using the more popular Public DNS services.

Now, we are working with our Datacenters to implement a stable DNS solution that should finally prevent these errors in the future. We expect a resolution within 24-72 hours (since it is a weekend now).

ShrinkTheWeb® (About STW) is another innovation by Neosys Consulting
Contact Us | PagePix Benefits | Learn More | STW Forums | Our Partners | Privacy Policy | Terms of Use

Announcing Javvy, the best crypto exchange and wallet solution (coming soon!)

©2018 ShrinkTheWeb. All rights reserved. ShrinkTheWeb is a registered trademark of ShrinkTheWeb.