Chevy Impala SS Forum - View Single Post - Here's what happened......
View Single Post
post #35 of (permalink) Old 08-07-2009, 05:05 PM Thread Starter
Founding Member
Join Date: Jul 2000
Location: Georgetown, Texas USA
Posts: 2,874
Here is the whole thing:
I am listed as Robert Allen since he set this up for me.

Posted By: Scott P. Date: Wed Aug 05, 2009 @ 06:20 am

It appears that not only did your server suffer from memory exhaustion, but its disk had been completely filled.
Since the disk was full, various services were unable to start, causing the website outages.
I cleared enough space to restart your server and bring services back online. I'm currently working to clean more space, and advice you of where most of your disk is being used.
--Scott P.
RackForce Support

Posted By: Robert Allen Date: Wed Aug 05, 2009 @ 06:44 am
Robert Allen posted:

Posted By: Scott P. Date: Wed Aug 05, 2009 @ 06:47 am

I discovered the following files on your server larger than 100MB:

6158M /usr/local/apache/logs/error_log
2048M /usr/local/apache/logs/suphp_log
1895M /var/lib/mysql/
1663M /usr/local/apache/logs/suexec_log
598M /var/lib/mysql/impalass_forum/post.MYD
500M /usr/tmpDSK
449M /var/lib/mysql/impalass_forum/post.MYI
415M /var/lib/mysql/impalass_forum/attachment.MYD
233M /home/impalass/tmp/analog/cache
136M /home/fbiss/tmp/analog/cache
134M /usr/local/apache/logs/access_log
Most of the files, the ones in italics, are just very large logfiles. Do I have your permission to clear these logs to save you more space?

--Scott P.
RackForce Support

Posted By: Robert Allen Date: Wed Aug 05, 2009 @ 07:13 am
Robert Allen posted:

Posted By: Tom S. Date: Wed Aug 05, 2009 @ 08:08 am

During the process of clearing the large files as requested a clerical error had occurred and in the process the file /var/lib/mysql/impalass_forum/post.MYD was deleted, our attempts to retrieve the file from the MySQL buffer were unsuccessful and we were unable to restore the lost data.

We sincerely apologize for the inconvenience this may have caused you and are willing to assist you in restoring any back up of this file you may have.

--Tom S.
RackForce Support

Posted By: Robert Allen Date: Thu Aug 06, 2009 @ 04:07 pm
Robert Allen posted:
Any idea how large that file was? I am seeing on my Forum that a lot of old posts seem to be gone. I really hate to think that the 10 years of data is gone.

Posted By: David V. Date: Thu Aug 06, 2009 @ 04:34 pm
Hello Robert,

Reviewing your server's information and the details from this ticket, I see the file was approximately 598MB, so that may indeed be quite a bit of data that was lost. We do want to extend our absolute sincerest and utmose apologies what has occurred once more, as we do recognize the seriousness of this issue and will be reviewing this matter internally.

In terms of recovery steps, while I do not see you are currently subscribed to our Tivoli backup service, I do see that you are running cPanel / WHM on your server. WHM can be set up to make regular backups of your server's accounts, and among these backups you can optionally choose to have MySQL databases included. At the moment it looks like the backup options are turned off, but do you know if your server was configured to make these backups before the deletion occurred? If so, it is possible the database could be restored from these accordingly. Failing that, it may also be prudent to check and see if you have any local backups of this file which could be recovered.

--David V.
RackForce Support

Posted By: Robert Allen Date: Fri Aug 07, 2009 @ 05:51 am
Robert Allen posted:
10 Years of DATA! I thought that the backups were automatic. Besides, if I am running out of room on my server, where can I back this database up to? Are you sure that the data can't be recovered?

Posted By: Shane B. Date: Fri Aug 07, 2009 @ 05:56 am

Unfortunately if you do not have local backups it appears we are unable to restore any of your data at this time. cPanel/WHM backups are not configured automatically, they need to be configured by the system administrator to have proper functionality.

As for your question in relation to where you could even store a backup when your drive space is running out, the simple answer would be to either remove old data to free up space or upgrade your hard drive to allow the storage of more data.

Is there anything else that we may assist you with at this time?

--Shane B.
RackForce Support

Posted By: Robert Allen Date: Fri Aug 07, 2009 @ 06:03 am
Robert Allen posted:
I think that I need to speak with a supervisor. Automatic backups were supposed to be part of the plan. I need someone to be accountable for this and make it right. 10 YEARS OF DATA!

Posted By: Shane B. Date: Fri Aug 07, 2009 @ 06:14 am

Certainly, you can contact the Support Manager by emailing [email protected] . I have escalated this to my seniors and they are aware of the issue, but if you would like further information please feel free to contact our Manager at the email address provided.

--Shane B.
RackForce Support

Posted By: Alex S. Date: Fri Aug 07, 2009 @ 11:20 am

My name is Alex and I am the support manager here at RackForce Networks. I have reviewed your account and it appears as if you have not had backups since April 2007. In the migration ticket (109627) that was created at that time to move your environment to a dds5000, no backups were selected. You have not had or been paying for backups since that time. Those backups have long since expired. We have no backups of your files currently. We sincerely appologize for this mistake however we have no way of recovering your data.

--Alex S.
RackForce Support

Posted By: Robert Allen Date: Fri Aug 07, 2009 @ 11:29 am
Robert Allen posted:
Your people set up the migration. That means that they should have migrated ALL of the settings? There has to be something that you can do. You don't just go in and delete SQL files!

Posted By: Alex S. Date: Fri Aug 07, 2009 @ 11:37 am

As stated before we simply do not have any method for recovering your data. You do not have backups currently. We do not have backups of that file. We offer our sincerest appologies however at this point we have no way to recover that data.

--Alex S.
RackForce Support

Posted By: Robert Allen Date: Fri Aug 07, 2009 @ 11:45 am
Robert Allen posted:
OK, We are talking about over 10 YEARS worth of data. Someone needs to be accountable for this loss. I guess that you can't just undelete it huh? Pull it back out of the trashcan? I will have a conversation with my lawyer and see what steps need to be taken from here. This data loss is irreplaceable, thousands of people across the globe use this site everyday as THE site for information about the Caprice and Impala.

Posted By: Alex S. Date: Fri Aug 07, 2009 @ 11:55 am

Please be aware of section 13 of our master service aggreement that states.

13. Limitation of Liability

Customer acknowledges and agrees neither RackForce nor any of its members, shareholders, directors, officers, employees or representatives will be liable for any special, indirect, consequential, punitive or exemplary damages, or damages (including but not limited to damages for loss of profits or savings, loss of data, or loss of use) in connection with this agreement. If, despite the foregoing limitations, RackForce or any of its shareholders, directors, officers, employees or representatives should become liable to Customer or any other person in connection with this agreement for ANY REASON, then the maximum aggregate liability of RackForce, its members, shareholders, directors, officers, employees and representatives for all such things and to all such parties will be limited to the lesser of the actual amount of loss or damage suffered by the claimant or the amount payable by Customer to RackForce for one month of service under this agreement.

I am going to cease conversation about any legal discusion and that will be taken care of by our lawyer should you choose to exercise that option.

We do deeply appologize and will continue to help with any technical problems you experience however will not discuss any legal matters via this ticket.
We unfortunately we can not simply "undelete" the data.

--Alex S.
RackForce Support

Posted By: Robert Allen Date: Fri Aug 07, 2009 @ 12:41 pm
Robert Allen posted:
1. I set up an account with you to host my site. Backup was included in the package.
2. Your people migrated my site to a larger sever. Backup was not migrated.
3. Your people deleted a 600mb SQL file instead of the log files.
4. I am now without anything.

Does that about sum it up?

Your employees gross negligence have now ruined my site.
No slap on the hand? Just an OOPS! Sorry about that?
Sincere apologies only go so far.

Please at least try to make this right. Do the right thing, and if you can't, bump me up to someone that can.

Posted By: Alex S. Date: Fri Aug 07, 2009 @ 01:54 pm

From a customer service standpoint we believe that your experience has not been what we would like it to be. Because of this we would like to offer a credit for a years worh of our Tivoli Backup Service to ensure such an issue does not occur again for any reason. It is our hope that this offer will illustrate that we do value you as a customer. While we are unable to recover the data that has been lost we can prevent this issue from happening again by ensuring that you are subcribed to a backup service.

--Alex S.
RackForce Support

Scott Williams - HACK
2S Manufacturing - Georgetown, Texas
Making cool stuff for the B-Body Community for 20 Years!
Impala SS Forum Administrator
HACK is offline  
For the best viewing experience please update your browser to Google Chrome