So a client came to us today complaining that their SharePoint site was dead. Looking at the performance monitor revealed that the processor was maxed out. SQL showed around 1300 active connections. IISReset and bouncing of the server proved to be of little help.
We shut the firewall off to the outside and were able to get the default page of the site to render without any problems.
After going back and forth troubleshooting the server for most of the morning, we took a look at the site's default page's web part maintenance page which showed a total of 88 web parts on the page, 54 of them being closed and in the Web Part Page Gallery. All 54 of those web parts were Content Editor Web Parts.
That's a lot of web parts. According to TechNet, the maximum recommended number of web parts on a page is 50. What the client had been doing was adding CEWPs to the page and once the content was no longer valid, hitting the 'X' on the web part, closing it, but not deleting it. Doing so caused SharePoint to still get the web part from the database, just not render it on the page.
According to MSDN documentation titled 'A Developer's Introduction to Web Parts':
A Web Part Page gallery contains Web Parts that are already added to the current page. This may seem paradoxical at first. Why would you need to add a Web Part that is already added to the page? The reason this is useful is that you can add a Web Part to a page and then close the Web Part. To close a Web Part, click the arrowhead on the right side of the Web Part title bar, and then click Close. A closed Web Part is no longer visible on the page, but it is still a member of the Web Part Page gallery for that page. The Web Part is still associated with the page by an entry in the configuration database of the server running Windows SharePoint Services, which also stores any shared or personalized property settings for the Web Part.
You can add a Web Part to a page, personalize it extensively, close it, and then later add it back to the page with the personalization intact. To bring back a closed Web Part, select it from the Web Part Page gallery.
Note To truly delete a Web Part from a page, you must click the Modify My Page menu (or the Modify Shared Page menu in shared view), and then click Design this Page to add the Delete command to the drop-down menu of the Web Part. To remove the Web Part from the page without adding it to the Web Part Page gallery, click the arrowhead on the right side of the Web Part title bar, and then click Delete.
After removing 50 of the closed web parts from the Web Part Page Gallery, not only was the site back up, but the response time for the page was faster than it has been for quite some time.