Strange things are going on.
Actually, today one of my most frequented servers, running lighttpd, stopped serving php files after working for months.
After searching for a while in the lighttpd logs and php logs we figured out, that lighttpd still serves plain html files. So the problem seemed to come from the fast-cgi php. After swichting to apache for about 30 minutes, apache stopped too.
So we figured out that there must be something with the db server, which is an external machine.
Connection local worked fine. Connecting via socket didn’t!
We looked at the processes with:
mysql>show processlist;
The result was a lot of these:
| 108160 | unauthenticated user | 10.0.0.1:3449 | NULL | Connect | NULL | login | NULL
So we figured out that there is a problem with the authentification. First we thougt it is a switch or some kind of hardware problem. Nothing.
After looking around, we found out that MySQL is using reverse lookup at socket connections.
We added the machines in /etc/hosts and everything works fine for now.
What happend?
We are not using our own DNS for lookups. So what I think is that there was actually an entry for 10.0.0.1 in the dns server we use. This dns server served some domain name. Definitly not for our server, but it was a dns record. Mysql just checks and says yes or no. Someone must have updated the dns server and deleted this record so our system stopped.
Strange thing!
I think there is no good reason why mysql should do a reverse lookup. Do you have one?