Updates to Thumbnail Capture generators

2 posts / 0 new
Last post
puravida's picture
puravida
Jedi Warrior
Offline
Joined: 09/01/2007
Visit puravida's Website

I have just rolled out an update to all capture generators AND put about 60,000 requests back into the queue. These are potentially marked as "broken", even when good. The generators are coming online quickly, so those should be processed in the next hour.

GEEK-SPEAK
Further testing revealed the cause of some inefficiencies in the system. We have been seeing alerts for months, since we launched the newest version of generators, but we had been unable to investigate until now.

It turns out that the generators were suffering from an operating system glitch that caused the render engines to fail under certain conditions. When that occurred, the generators were unable to report the exact failure and would terminate to preserve the integrity of the overall system. However, that increased the chance (by 50%) of the last request being marked as broken (even if good).

It also meant that the affected generator was unnecessarily offline for 1-2 minutes, while it attempted to run automated diagnostics and recover. By the time it recovered, time had been wasted. Overall, this was not a major problem but it did result in about 5-10% lost efficiency during peak demand and probably 0.5% improperly "broken" requests.

I have been monitoring the updated test gens for 24 hours and they have been extremely stable so far. Smile

puravida's picture
puravida
Jedi Warrior
Offline
Joined: 09/01/2007
Visit puravida's Website

The generators are working very well, but they are going through these requests much more slowly because so many actually are bad. Specifically, they are bad in ways that slow down the process and require retries. So it may be awhile longer. However, generators are still spinning up quickly to compensate.

ShrinkTheWeb® (About STW) is another innovation by Neosys Consulting
Contact Us | PagePix Benefits | Learn More | STW Forums | Our Partners | Privacy Policy | Terms of Use

Announcing Javvy, the best crypto exchange and wallet solution (coming soon!)

©2018 ShrinkTheWeb. All rights reserved. ShrinkTheWeb is a registered trademark of ShrinkTheWeb.