From Server Chaos to Data Safety: Recovering and Migrating a MariaDB Database

Dealing with server issues can be nerve-wracking, especially when critical databases like MariaDB are at risk. Recently, I faced a challenging situation where my server was unbootable, and the MariaDB root password was forgotten. With persistence and strategic troubleshooting, I managed to rescue the database, back it up, and migrate it to a new server. Here’s a detailed account of how I achieved this feat, ensuring no data loss along the way.

The Problem:

1. Unbootable Server: My Ubuntu server entered rescue mode due to kernel issues.

2. Forgotten MariaDB Root Password: Access to the database was blocked.

3. Critical Data at Stake: Backups needed to be created without compromising the integrity of the data.

Steps to Recovery:

1. Accessing the Server in Rescue Mode

Since the server couldn’t boot normally, I mounted the disk manually in rescue mode to access critical files. Rescue mode is a powerful fallback that lets you troubleshoot problems when a server is not functioning properly.

"mount /dev/vda1 /mnt chroot /mnt "

2. Restarting the MariaDB Service

MariaDB wasn’t running, and the typical service commands were unavailable in rescue mode. To manually start MariaDB:

Locate the MariaDB data directory.
Start the service directly using mysqld_safe.

" mysqld_safe --datadir=/var/lib/mysql & "

3. Resetting the Root Password

Without the root password, accessing the database was impossible. I reset the MariaDB root password as follows:

Start MariaDB in skip-grant-tables mode:

" mysqld_safe --skip-grant-tables & "

Open the MySQL shell:

MySQL

Reset the root password:

UPDATE mysql.user SET authentication_string=PASSWORD('newpassword') WHERE User='root';

FLUSH PRIVILEGES;

4. Dumping the Database

Once I had access, I exported the database using the mysqldump command to create a backup:

mysqldump -u root -p database_name > database_name.sql

5. Transferring the Backup to a Remote Server

To migrate the backup to a new server, I used scp:

scp database_name.sql user@remote_server:/path/to/destination

This securely transferred the database dump to the target server, ready for restoration.

6. Restoring the Database on the New Server

On the new server, I restored the database using the mysql command:

mysql -u root -p database_name < database_name.sql

This brought the database back to life on a fresh, stable server.

Lessons Learned:

Always Keep Backups: Regular database backups could save hours of troubleshooting.
Master Rescue Mode: Understanding how to navigate rescue mode is critical for disaster recovery.
Document Every Step: Documenting fixes ensures smoother recovery if the issue reoccurs.

Conclusion:

Server failures can feel overwhelming, but with a methodical approach, even complex problems can be resolved. By following these steps, I not only rescued my MariaDB database but also learned valuable lessons about troubleshooting and server management.

If you’re facing a similar issue, don’t panic. Stick to the process, and you’ll come out stronger on the other side.

Author

Hi, I’m Mohammad A., a highly skilled Linux Server Administrator, Troubleshooting Specialist, and Security Expert with over a decade of experience. Whether you're facing complex server issues, need a secure and optimized hosting environment, or require expert-level server management, I’m here to help.