Move MySQL data directory from one folder to another

Moving your MySQL data from one folder to another sounds like a big job, but in reality it is not really.
Before we get started you need to bear a couple of things in mind

  • Be sure to read this entire post before you do anything.
  • Especially read the part about the ‘socket’.
  • Be sure to understand everything that is in the post
  • Some versions of unix/linux use different terminology, (for example to stop/start a service), be sure that you use the correct command.
  • Be sure you are logged in as root.
  • Remember that there will be down time!
    The length of time depends a lot on the size of the data and where you are transferring to/from.
  • Be prepared to roll back the changes if things don’t work!
    • Stop the service.
    • Copy the original my.cnf file, (see below to locate it)
    • Start the service.
  • This is all done at your own risk! If you are not careful you could really trash your data!
  • Read the errors you might get, learn about your system, normally the error should help you solve most issues.
  • Try it on your test/dev server before you do anything, (you have a test server don’t you!!)

Locate the datadir.
This is where MySQL currently has all the data saved, everything is saved in one place.

Run the script, (with phpMyAdmin or something), below and look for the entry that says “datadir”.

# locate the current data directory
SHOW VARIABLES WHERE Variable_Name LIKE “%dir”

Locate the my.cnf.
This is the file that contains the MySQL configuration, this is a very important point and you need to do it carefully.

There could be more than one file so you will need to check all of them!!!

# locate my.cnf, (the very fist line been returned from the cmd below).
# more than one could be given
mysqld –help –verbose

This will return a whole lot of information, but the very first line will contain the locations.

  • Not all the files will exist
  • Maybe _none_ of the files will exist
  • If none exist, create a file called “my.cnf” in the first folder in the list.
  • If any of the files exist, look in any of them for the entry “datadir=…”
    (of course making sure it corresponds to the datadir you located in the previous step.

Stop the MySQL service.
This is a typical case where you need to choose the right command for your repo… do your homework first…

service mysql stop
# or maybe
# /etc/init.d/mysql stop

Copy the data from the old folder to the new folder

# copy the data over, (change the directories).
# rm -rf /home/mysql/*
cp -R -p /var/lib/mysql /home/mysql

You must learn what this means, “cp -R -p” means, copy the data from the source folder to the destination folder, (“/var/lib/mysql” and “/home/mysql” in my case).

But what this also does is keep the permissions of the folder, normally a user “mysql” is created and has full permissions to that folder.

For things to keep working, you must make sure that the permissions remain.

Spend a bit of time making sure that …

  • All the files were copied
  • All the permissions have been kept.

Edit the datadir.
In the my.cnf that you edited/created earlier change, (or add), the datadir.

[mysqld]
datadir=/home/mysql
#innodb_data_home_dir
#innodb_log_group_home_dir=/home/mysql/

  • There could be other values, only edit/add the datadir=, don’t change anything else!
  • In your first step, where you located the directory, look for …
    • innodb_data_home_dir=, if the directory is the same/similar to the one you just moved, then either add an entry or edit the one that is there with the same folder.
    • innodb_log_group_home_dir=, same as above.

Start the MySQL service.

service mysql start
# or maybe
# /etc/init.d/mysql start

Immediately make sure that the server is up and running

mysqladmin status
# or using phpmyadmin# or using whatever tool you fancy…

If it is running ok, make sure that the directories are indeed valid, (same step as earlier).

# locate the current data directory
SHOW VARIABLES WHERE Variable_Name LIKE “%dir”

If the server is not started you might get a ‘socket’ error, in that case, look for where MySQL thinks it is…

SHOW VARIABLES WHERE Variable_name = “socket”

It might point to the old directory, in that case you need to edit the config one more time.

But first stop the service…

service mysql stop
# or maybe
# /etc/init.d/mysql stop

Add two socket section, (one for the client and one for mysql itself)

[client]
# …
socket=/home/mysql/mysql.sock

[mysqld]
# …
socket=/home/mysql/mysql.sock

Of course, in my example, I am using my directory and I am only showing what needs to be edited/added.

Restart the service…

service mysql start
# or maybe
# /etc/init.d/mysql start

And make sure that it is all good!

Delete the old data

If everything is working I would suggest that you back up the old data somewhere else.

To make really sure that all us good, simply rename the old folder, (don’t move it or anything, just rename), and make sure that things are working).

Apache webserver error on startup

I don’t use Apache that often, (not as often as I used to), but when I do I sometime get an error telling me that

(OS 10048)Only one usage of each socket address (protocol/network address/port) is normally permitted. : make_sock: could not bind to address 0.0.0.0:443 no listening sockets available, shutting down Unable to open logs

Now in my case the error was because of skype.exe that was using port 443

But if you need to know who is using it simply open the command line and type

netstat -ano

And, somewhere in the list, you will give see something that is using 0.0.0.0:443, but you will not see the name, you will see the pid, (the process id).

BTW: to get the error message, you can get it via the logs or you can open a command line and run the Apache server manually.

httpd.exe -k start

What I had forgotten about WM_CLOSE

When I was working on the next version of Piger I would sometime get an issue when using the this.bye command, (to close Piger).

What happens in the background is I call WM_CLOSE to close the active window, this message is sent to all the windows.

When it is processed by the message pump no other messages are been processed, this is a problem in case some windows are still dishing out messages, (like a fading dialog box in our example).

So remember, once you call WM_CLOSE, all bets are off and your message pump is as good as dead.

Best to have your own ‘Close()’ that does all the housekeeping in the background before you actually post WM_CLOSE to the main thread…

Otherwise your app my appear to hang.

Preventing embedded python from killing your app

I was looking at a defect in Piger where a Python script could close Piger itself, it wasn’t a crash, but rather a graceful exit.

import am
am.say( "Bye", 1, 1 );
exit(1);

The problem was with the way I was calling the script rather than anything wrong with Piger or Python itself.

Somewhere in the Python virtual machine I had something like…

...
if( -1 == PyRun_SimpleString( ... ) )
{
...
}
...

And, while that works fine in most cases, if I have an exit( xyz ) in my script, piger would close.
So the answer was to replace it with …


...
PyObject * PyRes = PyRun_String(s, Py_file_input, main_dict, main_dict);
PyObject* ex = PyErr_Occurred();
if( NULL != ex)
{
...
}
...

The one last problem was, how to tell if the error was because of an exit(…) or because of an error.


...
PyObject * PyRes = PyRun_String(s, Py_file_input, main_dict, main_dict);
PyObject* ex = PyErr_Occurred();
if( NULL != ex)
{
// if we exit(...) then 'ex' is not NULL, so we must check for that.
if (!PyErr_ExceptionMatches(PyExc_SystemExit))
{
// this is a real coding error.
}
}
...

Now you can catch all the real errors and swallow all the exist code and so on.

See the exceptions handling in Python and embedding in general.

Released piger 0.3.2

I released the first version of Piger on Github after moving from sourceforge.net

This new release is pretty much the same as the previous one just brought to the 21st century …

The changes are

  1. Python 3.5.1
  2. Python is not required on the host machine, (something that was causing a break before!)
  3. Lua 5.2
  4. x64 or x86 build, (not for memory reasons, but for env. variable reasons).
  5. Moved from Sourceforge.net, (I added a readme.md to redirect to github). Please Google for some well documented reasons to move away from them…

Please try it and tell me if you see something cool that should be added.

I am also looking for cool scripts to be added to my list, so please, send them my way!

Embed Python in your C++ application without Python installed on guest machine

For my Piger application, I wanted my C++ app to parse Python scripts but I quickly became aware of the fact that the user must have Python installed on their machine, not only that they needed the same version as mine, (version 3.5).

The normal call would be something like…

#include <Python.h>
...
// This will cause an error if the user does not have the same version of Python as the one we want.
// or if they do not have Python installed at all.
Py_Initialize();
...

But that throws an error when the user does not have Python installed.
And if they have an older version installed, it _might_ work, but the scripts might not work as expected.

The solution is to embed the Python you want to run in your app.

  1. Go to the Python website and download the Embeddable zip file for your app, (x64 or x86)
  2. Extract the python35.zip located inside that zip file
  3. Copy it somewhere where it can be referenced.
  4. Add the code below.
#include <Python.h>
...    
std::wstring exe_dir = L"\\exe\\path";
std::wstring python_path;
python_path += exe_dir + L"python35.zip";
Py_SetPath(python_path.c_str());

// Now we can call Initialize
Py_Initialize();
...

Now your app will work using version Python 3.5.1

Installing Eclipse C++ on Windows

The steps below are for installing C++ on eclipse and window.

I am using Eclipse 4.5 and Windows 10, but it doesn’t really mater, it should work for any version.

  • First things first, close everything, whatever version of Eclipse you have open.
    To make your life easier, close things you do not need as well, like notepads and so on.

MinGW

  • Go to mingw and download it, (MinGW stands for Minimalist GNU for Windows).
    The version I got was version 0.6.2
    Go to the website, (link above), and click on the “Download Installer” button, (it is not obvious at first where the button is).
  • Run the install as an Administrator.
    Pay attention to the install folder, (For me the default was C:\MinGW).
  • Wait for the install to complete …
  • When this is all done, add the ‘bin’ folder to your PATH environment variable, (Google how to do it), in my case I added “C:\MinGW\bin” to the PATH.
  • Open a command line window and type ‘mingw-get install gcc g++ gdb mingw32-make‘.
  • Wait for the install to complete …
  • Check that everything will work type “gcc –version“, if you get an error, something went wrong.
    Otherwise you should see the gcc version number.

Eclipse

  • Open your version of Eclipse
  • Go to Help > Eclipse Marketplace …
  • Type C++ and select the CDT option
  • or add “http://download.eclipse.org/tools/cdt/builds/mars/milestones” as a software site and select CDT main features.
  • Download and Update everything.

Others

  • Make sure that the right tool chain is selected
    • Project > Properties …
    • C/C++ Build
    • Tool Chain Editor and select “MinGW GCC”