Migration Final Phase Launch (downtime possible)

6 posts / 0 new
Last post
puravida's picture
puravida
Jedi Warrior
Offline
Joined: 09/01/2007
Visit puravida's Website

The time has finally arrived for us to complete the final phase of migration. This has been a long, arduous journey and we appreciate everyone's patience as we focused for the past couple of years to get here.

What to expect
We expect no actual service downtime (e.g. no response from the service; broken images; etc). However, the "offline" responses will be shown until we are fully migrated. That process, if all goes well, will take 30-60 minutes. If there are problems, we could push up to 2-3 hours and would, in a worst case, have to revert back to the current system until a future attempt.

However, we have tested very, very thoroughly on the new system and find it to be extremely solid. Therefore, we are expecting no customer-facing issues.

For further details on what we hope to achieve, please see the Massive Migration Project post.

When to expect it
I apologize for the short notice, but we cannot afford to delay any longer and must finish the migration as quickly as possible. Therefore, we are planning on launching at 02:00am ET (GMT -0500) and are preparing for a 2-3 hour offline maintenance window.

Should we need to push that time back, I will update this thread but not send a new notice.

puravida's picture
puravida
Jedi Warrior
Offline
Joined: 09/01/2007
Visit puravida's Website

Something not mentioned in those posts is that we are finally able to overcome various limitations of the previous system design.

We will finally be able to:

  • fully support all valid UTF-8 requests & encoding.
  • have a more uniform method to store requests, so that calling a request using different encoding will not require a new capture (saves time).
  • optimize some settings that will shave precious seconds off of each capture request.
  • simplify, normalize, and optimize thumbnail history searches.
  • consider a purely capture-only service offering.* (not yet available; only considering)

*This would be something along the lines of using our service to capture screenshots via dedicated generators but have them uploaded directly to your cloud file storage or FTP account. This would:

  • bypass the need to pay us for bandwidth and storage costs.
  • make overall costs much lower and capture times more predictable.
  • pave the way for having "customized" dedicated generators that could potentially capture in specific ways not available to other users.
  • also completely isolate an account's requests from the other requests, for those who require security assurances.

There is more but it is behind-the-scenes stuff that is boring but will eventually allow us to reduce pricing and offer improved service.

puravida's picture
puravida
Jedi Warrior
Offline
Joined: 09/01/2007
Visit puravida's Website

It looks as though everything migrated fairly smoothly.

There were only a few obstacles that delayed longer than I expected, but we still went LIVE within the two-hour time frame. The system was operational within 1.5 hours of beginning the migration, but all external requests were failing until right about the 2 hour mark, which is when I realized what was going on and fixed it.

So after more than 2 years of coding, planning, testing, and preparation; we can finally put this project behind us. Hooray!

We are not done, however, as there are always things to fix, improve, or optimize; but this is a BIG step forward. I hope that the migration did not affect anyone too much.

GEEK-SPEAK
THE OBSTACLES

The database restores had several issues, mostly a small configuration issue that we could not test beforehand and also a conflict caused by database usage during restore. Once we disabled all access and fixed the configuration issue, everything restored as expected.

Since I didn't realize the slight IP difference on the images.shrinktheweb.com DNS entry, it was not updated when it should have been. So most requests were failing against the old server for about 45 minutes longer than necessary. Sorry!

We also ran into an issue with launching the additional capture generators that are required with "starting over" with all new captures. We have alerted Rackspace and hope that they will provide answers to avoid the issue in the future. However, at least the old method worked, albeit at a higher cost to us. Once the influx of new requests subsides over the next few days, we will swap out the costlier servers with the newer, less expensive ones.

Also, we are now running on the new storage platform that is infinitely scalable, more reliable, and supposedly more stable. In our volume testing, we saw no errors; but once we went LIVE, we have seen a couple of requests fail to store properly. This is a little concerning, but I think that we can implement some "retry logic" to mitigate the occasional failures and, hopefully, our storage solution vendor will be able to get a handle on the failure causes soon.

buyersg3
Offline
Joined: 11/19/2012
Visit buyersg3's Website

My directory was set up only a month ago, and was mostly working fine with free service.
Today about noon 12:10 pm, the thumbnails started failing all over the 65 pages.
Says thumbnail queued.
I've seen that before when one is new, but this is now spreading to be 2/3 of the 800 images.

Is this due to the changeover? is there a backlog?

CORRECTION: OK it seems to be mostly, not only those links added in the last week or so, which possibly were not all seen before. But there is no update feed of those queued.

And it is not 2/3, maybe 1/4. Sorry. But I did see a few revert back to queued, perhaps to status of a few days ago, and still no feed.

puravida's picture
puravida
Jedi Warrior
Offline
Joined: 09/01/2007
Visit puravida's Website

As announced as part of the "Massive Migration Project," we have wiped the entire database and started over. All requests are considered new and being recaptured.

There are so many generators online that requests are coming back rather quickly, so that is why they were all queued one minute and then a fraction the next.

puravida's picture
puravida
Jedi Warrior
Offline
Joined: 09/01/2007
Visit puravida's Website

Also, a quick reminder...

This migration enforces a few lingering security issues that have been allowed to go on for quite some time. These changes were announced many times over the past couple of years and now they are strictly enforced.

In short, they are:

  1. All requests must use
    • images.shrinktheweb.com/xino instead of
    • www.shrinktheweb.com/xino
  2. All "simple method" embedded requests must conform to the security standard of NOT passing the "secret key" in the request. Therefore, you must use &stwu for "advanced method" ONLY. Otherwise, be sure to omit it.

There may be other changes. I will update this list as I have time to go back and review notes.

ShrinkTheWeb® (About STW) is another innovation by Neosys Consulting
Contact Us | PagePix Benefits | Learn More | STW Forums | Our Partners | Privacy Policy | Terms of Use

Announcing Javvy, the best crypto exchange and wallet solution (coming soon!)

©2018 ShrinkTheWeb. All rights reserved. ShrinkTheWeb is a registered trademark of ShrinkTheWeb.