Recap of Recent Improvements & Issues

1 post / 0 new
puravida's picture
puravida
Jedi Warrior
Offline
Joined: 09/01/2007
Visit puravida's Website

Sample PHP API Library Update Released

Due to reported confusion on how to configure a sample request and my discovery that the antiquated request option logic doesn't work in several cases, I have published a major overhaul to the sample PHP API library (now v2.1.0). This fixes a known HASH_MISMATCH error condition caused when passing certain types of invalid data in certain parameters.

Unfortunately, this overhaul was so extensive and I needed to remove deprecated code, so this version invalidates existing filenames of cached screenshots. While I always try to keep every change or new feature backward-compatible, I have avoided updating this logic for years, and now it has become a burden and made the code difficult to read. This release greatly simplifies basic functions, which are now much more intuitive; making it easier to make changes and customizations. IF you are running on an older version and decide to upgrade to v2.1.0 of the STW API Library, please note that we strongly recommend that you clear the screenshot cache and let it rebuild automatically. However, some will hit daily or monthly limits due to clearing the cache. In those cases, please contact me for a temporary allowance to overcome the limits.

Visit the ShrinkTheWeb API Documentation to download the latest PHP API Library.

Numerous Security-related Changes

The update I pushed today has finally provided the ability to loosen security a bit, but there are caveats to consider.

1. The sheer number of changes in this update affects SSL security & performance, web & service header security & performance, and combines several new features, bug fixes, and performance enhancements. While these improvements should be great for everyone, it also means there is risk of issues missed during our extensive testing. Please report any unexpected issues.

2. This update closes a long-time loophole that allowed scripts to bypass "Referrer" validation, in some cases. These cases tended to be uses such as distributed mobile applications or browser extensions using the "Embedded" (or "Simple Method") API. While we do want to be as flexible as possible to cater to everyone's creative uses for this service, we must also consider security ramifications of that flexibility. Now that the loophole is closed, some applications may stop working as before (possibly for years!). Now, it will be required to upgrade to become a paying customer. Any upgrade will allow "Embedded" requests from any IP, and the system's default behavior will be to detect this type of application and automatically allow access to end-users. In case of issues or suspected service theft by this method, there is a new security option "Never Allow Automatic IP Override" which, when selected, will prevent access from these applications, except when the IPs are specified in the "Allowed Referrers" list.

3. Now, all new users will be given 32-character SECRET keys. Existing users will keep their SECRET key, in order to avoid "breaking" existing integrations. However, users may login to the ShrinkTheWeb portal and update their SECRET in the Security area. Unfortunately, I was unaware that WP-Portfolio was hardcoded with a 5-character limit from before we took over maintenance of the software. So, all new users will be unable to enter their credentials to use WP-Portfolio with ShrinkTheWeb, unless they download the latest WP-Portfolio v1.40, once it's released this week. In the meantime, please open a support ticket.

Users Unintentionally Blocked

After investing more than $250,000 in time, money, and resources into building out ShrinkTheWeb on top of an "infinitely" scalable, auto-healing infrastructure; I have discovered an oversight that caused thousands of users to be unintentionally blocked from service access. This has been corrected and added to our list of things to review when testing changes.

One of the effects of this issue was that thousands of accounts were unexpectedly "disabled due to inactivity" over the past 24 months. Please request reactivation, if this affected you.

This issue also brought to light an occasional issue with WP-Portfolio whereby the screenshots would all begin showing the "Error Thumbnail" (for BOTH embedded or cached locally images). I have a great developer, who helps with WP-Portfolio, looking into this issue now.

New Pricing Model & Site Re-launch Delayed

Due to the mountain of tasks required to fully automate this service, while also building it out on the new infrastructure; I was convinced to outsource the migration of the website, user portal, and API. The developer I've been working with over the past 6 months is really sharp and produces great work, but unfortunately, his proposal to migrate to a certain framework did not pan out. After spending 80% of the migration budget and working for 3 months, he notified me that the framework would not do what was required without significant effort. As sometimes happens in IT, the project hit a major obstacle that would delay by a year or more and could cost 10 times the original budget; so I have to cut my losses, start over, and do it myself now. So, the site which the new pricing model is waiting on, will be delayed by several more months. Sad

Speaking of pricing, if you would like to give feedback in a public discussion on pricing, please visit the ideas forum: ShrinkTheWeb Revised Pricing Model (planned for 2017)

-----

On a positive note, the service has been running faster than ever, despite optimizations that reduced expenses by a HUGE amount, across our hundreds of servers!! This savings is especially evident under heavy load, such as bursts of 1 million captures in a short time, which scales up to thousands of capture instances for a fraction of the previous cost. This is great news for users, because it means that my recent HUGE investments into infrastructure have paid off to such a degree that it makes the service even more viable under continued growth and paves the way for keeping prices low for many users. Hooray! Smile

ShrinkTheWeb® (About STW) is another innovation by Neosys Consulting
Contact Us | PagePix Benefits | Learn More | STW Forums | Our Partners | Privacy Policy | Terms of Use

©2017 ShrinkTheWeb. All rights reserved. ShrinkTheWeb is a registered trademark of ShrinkTheWeb.