So, essentially, we don't know much. At this point, we're guessing that the server was shutdown for a kernel update, since the kernel version has changed.Jul 26 11:33:15 s15353558 shutdown[16194]: shutting down for system halt
Jul 26 11:33:15 s15353558 init: Switching to runlevel: 0
Jul 26 11:33:15 s15353558 saslauthd[26087]: server_exit : master exited: 26087
Jul 26 11:33:23 s15353558 xinetd[25725]: Exiting...
Jul 26 11:33:24 s15353558 kernel: Kernel logging (proc) stopped.
Jul 26 11:33:24 s15353558 kernel: Kernel log daemon terminating.
Jul 26 11:33:25 s15353558 rsyslogd: [origin software="rsyslogd" swVersion="2.0.0" x-pid="25678"] exiting on signal 15.
Jul 26 14:41:09 s15353558 rsyslogd: [origin software="rsyslogd" swVersion="2.0.0" x-pid="22478"][x-configInfo udpReception="No" udpPort="51
4" tcpReception="No" tcpPort="0"] restart
We don't know why the database wasn't cleanly shutdown - It could be a timeout function on the host, where it kills seemingly non-responsive process after x seconds.
What we do know is this:
1. The file for mysql that was storing the topic list is simply gone. Most likely suspect is that ReiserFS did not get to write everything to disk, and either didn't replay the journal properly, or lost the file reference. If we could have mounted the VPS disk and run a scan, chances are, we would have found the original file. However, without having access to a KVM/any other partition, this route is impossible.
2. The automatic backup worked once, and then didn't work. Exact cause of the failure is unknown, but I'm hoping the daily backup thing (as part of phpBB) will work. If not, mysqldump in a cron script will be used.
3. Not all the data in the topic list could be recovered. We can guess at which forum, what the title was, who posted it, and when it was posted because most of the stuff can be derived from the first post in each topic. Behind the scenes, there's a lot of stuff being stored, and thankfully, it saved us in this case.
Any questions, or is that satisfactory?
