We had a bit of a time discovering why our project's Zend Framework-using PHP application wasn't working at all well in our production environment. The first few page loads were working fine, but concurrent page loads were presenting a problem. Neither PHP nor Zend Framework were at all helpful in trying to track down the problem.
It turns out that the application was misconfigured to store its Zend Cache files on a NFS mounted remote file system, and that Zend_Cache_Backend_File tries to flock() its files. On RHEL5, the NFSv3 implementation doesn't react at all well to POSIX locks.
We cracked the problem by figuring out that the locks seemed to timeout at the NFS timeout period, while the rest of our timeouts were at one-minute range, using pstack to dump the stack trace of hanging Apache processes, finding flock() to be the culprit, and grepping through our app's source code for flock, finding only the Zend Cache classes.
Don't accidentally put your Zend Cache files on NFS when using Zend_Cache_Backend_File, or your application will hang.
1 comments: