Thursday, October 22, 2009

a word about QuestionPoint service interruptions

This week I have had both a QuestionPoint Users Group and a 24/7 Reference Cooperative Advisory Board meeting and I received further clarification as to the issues that QuestionPoint has been experiencing the last few months:

On June 21st, QuestionPoint moved to a larger capacity back-end database and new operating system. This was a move initiated by OCLC for all their products because of increasing traffic and the need for higher capacity.

In September, traffic increased as per usual with the start of the school year and this stressed the new database application. There were tweaks and optimizations completed to manage the issues occurring but it wasn't enough. Throughout the last week of September, there were many "bounces" to the server. When QP "bounces" the system what they're doing is taking the server offline and immediately rebooting it in order to clear out the cache and database tables that get bogged down. On average, a "bounce" takes 10 - 15 minutes to complete before the system is up and running again. Often, Cathay and I do not get a lot of notice before a "bounce" occurs but if you are unable to access QP, try again in a few minutes.

September 27th, QuestionPoint moved to a previous version of the new back-end, high-capacity database software and saw a significant reduction in connection and server issues. For whatever reason this move also caused the data counts for the 9/27 daily reports to quadruple. The monthly data counts for September are reported to be correct but the 9/27 daily count is still incorrect.

Currently, response times are mostly back to pre-June levels, sometimes better. There are occasional surges (like today's problems) but the QuestionPoint team is tirelessly working to identify and resolve these issues. Both Cathay and I and the QuestionPoint team thank you all for your patience while these issues are sorted out.

Cathay and I do our best to keep the Staff Support website and mdaskusnow listserv up to date when we know issues are occurring but the best and fastest thing we can do is update our status messages in our Meebos on the Contact page. You can also use these to contact us in case of an issue. If you were in the habit of checking the OCLC system alerts page I learned that the page is updated by OCLC support and sometimes takes a while for QP to get an alert out; they usually forgo this if they believe it's a "short" issue. Instead, they rely on messages to the QP_L list, which we then fwd to the mdaskusnow listserv and note on my meebo.

When customers try to access AskUsNow! and QuestionPoint is down, they see an OCLC page letting them know that the system is down. If they are already in a chat session and the server goes down, they are disconnected; unfortunately in this manner they may not realize it is a tech problem versus anything else. We do have a note on our Welcome Login page that customers see when the come into the service that lets them know we are having difficulties. Hopefully this message helps alert people to the possibility of connecton issues.

Again, we thank you for your patience and hope that we are able to keep the frustration down to a minimum. Please feel free to contact us if you have any questions about this.

No comments: