Strange problems with Symfony2-based project
Роман Лапин — Feb 24, 2012 development
That’s gonna be our first post in English here. The reason is that I personally believe that it might help someone in case they meet with those same strange issues we ran into recently.
In November of 2011 we started new project for our client and chose Symfony2 as a base framework. First of all we really wanted to try it in real life as lately there are a lot of things happening in PHP community around Symfony2 and related projects. Also we had a very limited amount of time to bring this project to life. And gethering information about different frameworks we decided that Symfony2 would be the fastest in means of speed of development.
As for latter, it happened to be absolutely true. Working with Symfony2 is great pleasure and fun. We developed the base functionality in about 1.5 moths and then started to clear out bugs and add new features. I think we’ll describe some of the moments about development with Symfony2 in nearest future.
Couple of weeks ago site replaced old one on it’s main domain and strange things started to happen. Our logs started to fill with occasional “non-object” errors with stack traces in framework code and site just returned blank pages on them. We started to experiment and researching. Our first mistake was that we placed this project on Amazon EC2 Micro intstance and haven’t read about CPU resources on it. Symfony2 appeared to eat CPU quite extensively. We switched fast to Small Intstance, CPU graphics was fine there but errors continue to appear repeatedly.
After some time I posted a question on stackoverflow hoping that anybody met this kind of bugs. Thanks for everyone who tried to help with ideas.
We made all kind of possible logs, tried project with different APC settings and without APC at all. Also we tried both apache2+mod_php and nginx+php-fpm. Ryan’s suggestions about memory issues was most possible variant, playing with APC didn’t help, but it give me the idea and we started to scan bugtrackers and chagelogs of all pieces of our server software stack for memory and non-object issues.
Finally, we just upgraded PHP on the latest server that we tried to the latest version 5.3.10 replacing latest version available from aptitude. And error flow just stopped. Actually, I don’t no why it was not the first thing that came to our minds.
Here I want to say special thanks to our big friend and good specialist Ilya, who assisted us with his help during all time we were fighting with the problem. Actually it was him who gave us one more server to test site and it was him who just upgraded PHP and everything became just good.
This situation gave us a good lessons: both in business and technology. During those days I learned a lot of things about server configurations and interactions that I couldn’t imagine I would ever learn. From business point of view we learned, how to explain bad situations to client. But the main conslusions are of course about all sort of possible improvements that we’ll introduce to our development, deployment and testing processes to avoid same situations in the future.
Thanks for your attention. Hope you’ll never meet any kind of Heisenbugs!