w3wp.exe using too much memory and resources
w3wp.exe using too much memory and resources
Posted on Oct 1, 2009 by Paul WhiteIf you are here, the chances are you have already gone through a few pots of coffee, and lost a few hairs over trying to figure out why your W3WP.exe process is consuming so much memory. The solution is actually very simple, but first a little background information.
What is the W3WP.exe process?
The W3WP.exe is the IIS worker process, that handles requests, maintain sessions, viewstate, and the cache. Its been around Since IIS 6 on Windows Server 2003, and is still found as part of IIS 7.5 with Server 2008 R2. On a properly configured, and programmed website the W3WP.exe can usually stay under 100 Megs. If you are running a 64bit OS it could be a little larger. The actual size of the W3WP.exe will be determined by how much traffic your website gets, and how many pages it serves. The 100 Meg figure is just an estimate from my own experiences. Application Pools with multiple websites can easily go over this figure, so don't use it some magic number. But if you are running a website that has relatively low traffic ( under 20K impressions daily ), and your W3WP.exe is taking up 100+ MB of Memory, then you might want to make some changes.
Why do I care if W3WP.exe is large?
Lets say that you have a website running on a shared hosting account. As your site grows, it will start to experience bottle necks from the limited resources it has access to on the shared box. Remember the average shared server is running on average 300 websites. You site is constantly fighting other sites for CPU time and resources. Eventually you out grow the shared environment, and are forced to upgrade to either a VPS or a fully dedicated server. But what you don't know is your site could last much longer on the shared box or that small VPS, if you were to clean up your code.
Most popular causes of W3WP.exe using too much memory
Page Caching Dynamic Websites
Some Programmers will use Page Caching on their ASP.NET pages to improve request speeds. Even though this does work. It will quickly eat up all your memory. For Page Caching to work you would need to cache each page and every combination based on its querystring variables. For Members sites this could result in millions of different pages. A more effective way of using caching is to Cache objects that do not change, and are used throughout the website. Things like your menu bar. Or maybe the details for an event. This will reduce the number of calls to the database server, and speed up your site. Object Caching can make a huge difference on page load times, especially when you are able to cache objects that use expensive queries. If you want to learn more about how to custom manage objects in the ASP.NET cache, I wrote a few helper functions that are easily integrated into your websites.
ViewState Enabled on Labels
The ViewState is how ASP.NET keeps track of changes that have been made to Web Controls. ViewState is needed anytime you are going to do a postback, like for a submit form. But a common mistake is programmers don't disable viewstate on Label controls that are often used as place holders for long concatenated HTML strings. If you suspect your may be guilty of this all it takes is a look at the HTML source code from your web browser to confirm it. If your HTML source looks like this
with a really big view state then you need to disable the viewstate on your label controls. The Viewstate is stored in the W3WP.exe process. Its easy to see how this process can consumer so much memory when you have a view state this large and on an exponential number of web pages. The solution is to change your ASP:Label Controls from this
<asp:Label ID="mainmenu" runat="server" />to this
<asp:Label ID="mainmenu" EnableViewState="false" runat="server" />Lets face it there is no reason you need to preserve the ViewState of a Label Control. So go through all your pages and add the EnableViewState="false" attribute to all your Label Controls.
Long Session Time Outs
Anytime a visitor ( organic or spider ) visits your website, ASP.NET is going to maintain a session with them. Sessions are how the server keeps track of your users and their activity on your website. Without Sessions things like postback would not be possible. When ASP.NET uses sessions in the traditional way, they store a Session ID in a cookie on the client's browser. Then every time this client communicates with the server this cookie value ( Session ID ) is passed along with the request. The Server can then identify the visitor by their Session ID and handle the requests appropriately. Now the server has to keep track of the visitors activity. So each time a submit form is submitted, or a web control is used, the information related to this action is stored with the viewstate in the W3WP.exe, and is tied to the Session ID. This memory is not released until the Session Expires. The default on ASP.NET is for Session to Expire about 20 minutes. This means after 20 minutes of no activity from this visitor the Session would be discarded, and the memory associated with this session would be freed. Now if you have some lengthy submit forms which could take longer than 20 minutes to fill out, then you might need to have a longer than normal session timeout. On blog.whitesites.com I have sessions set to 45 minutes ( sometimes I take a while to write an article ) But if your submit forms are the quick kind that can be filled out within a few minutes, then 20 minutes is more than enough. If your website doesn't have any submit forms, then it might make sense to take your session timeouts down to 5 minutes or less. The shorter your Session Time Outs, the less memory that will be needed to track your visitors as they navigate through your website. If you want proof. Set your Session Timeouts to be 6 hours. Then watch over the next 6 hours as your W3WP.exe grows to a massive size. I can only imagine what sites like Facebook must deal with considering their visitors ( or should I call them addicts ), do daily surfing marathons.For a website like Blog.whitesites.com, in which the only public web control is the comment box at the bottom of the page, I could probably get by with a 15 minute session timeout. But my backend that is run from the same Application needs much longer Session timeouts. If I wanted to get very efficient with my memory, I would create a separate application for my backend ( control panel ) with the longer timeout, allowing me to shorten the timeout on my public application.
So in summary. Lots of submit forms, plus lots of unique visitors, plus a long session time out will equal a very big W3WP.exe. You might not be able to change your submit forms, or your visitors, but you can shorten your session timeouts.
Memory Leaks
Certain objects should be cleared from memory with the Dispose() Method. These include things like BitMap Objects. Even though the ASP.NET Garbage Collection does a good job of taking out the trash, I have found that on heavily loaded systems if you don't release Bitmaps with the Dispose Method it tends to lock the file image file you were working with. This object will sit in your W3WP.exe taking up space. Even though memory leaks can be a problem, they are usually very minor. ViewState is by far the biggest Memory hog.
Unclosed Database Connections
If you use helper functions to manage your Database connections, be sure that you close the connections, when finished. Else this will waste memory on both the W3WP.exe side and the Database Server side.
Conclusion
Even though advanced programmers might feel some of this is obvious. I found it disturbing that I was unable to find more articles online involving something so important. Who knows, maybe if every programmer uses my advice, this could negatively affect the bottom line of hosting companies, who's customers won't be needing to get that dedicated server just yet. If this helped you let me know.
Readers Thoughts
I have never used sharepoint before, though it seems that most medium to large scale companies have it running. It doesn't really surprise me that Sharepoint is a memory hog. From what I have seen in the development world the server-sided technologies that require the least effort from a programmer, usually have the most chub. Most developers would aggree that Linux with apache is the most conservative on memory usage, while asp.net can start to get a little greedy especially if you don't know what you are doing. Then you have platforms like Cold Fusion that are just plan Fatties. I am still a fan of ASP.NET. If you know how to properly manage your memory and objects it can be extremely lean and fast.
I have two Windows 2008 R2 servers up and running .Net web apps. As
time goes by (2 weeks), we have seen the Memory usages go up as high as
5Gigs. The Developers are really nervous with this kind of usage. How do
I explain it to them that this is normal for the 64 bit platform?
This application was 32 bit and we upgraded from Windows 2003 32 bit to
Windows 2008 R2 64-bit. The servers are only web servers running IIS 7.5.
Jesse,
First I would check to make sure the memory usage is from the W3WP.EXE files. One thing is if you are setting up the websites in the traditional way, each website / application gets its own App Pool. I would look into how much memory each website is consuming, to try to find out which one is taking up all the memory. going up to 5GB on a 64 Bit Server is not bad, unless you are running a little low on memory. Someone I do with all my applications is I schedule them to recycle the App Pools every 24 hours. For an Application to grow over 2 weeks tells me that the Application is not set to recycle after X minutes. If you goto the advanced properties on the Application Pool you will be able to change the recycle interval. However I would check your applications using the tips I listed above. Also check to see if your developer has page caching enabled. If these are dynamic sites this can quickly consume a lot of memory. Hope this helps
First I would check to make sure the memory usage is from the W3WP.EXE files. One thing is if you are setting up the websites in the traditional way, each website / application gets its own App Pool. I would look into how much memory each website is consuming, to try to find out which one is taking up all the memory. going up to 5GB on a 64 Bit Server is not bad, unless you are running a little low on memory. Someone I do with all my applications is I schedule them to recycle the App Pools every 24 hours. For an Application to grow over 2 weeks tells me that the Application is not set to recycle after X minutes. If you goto the advanced properties on the Application Pool you will be able to change the recycle interval. However I would check your applications using the tips I listed above. Also check to see if your developer has page caching enabled. If these are dynamic sites this can quickly consume a lot of memory. Hope this helps
Thank you so much for replying to my question. We only have one web application on that web server. When I go to Advanced settings under the app pool for the web site, I see many different parameters available for Recycling:
Disable Overlapped Recycle = False
Disable Recycling for Configuration Changes = False
Private memory limit = 0
Regular time interval 1740
Request limit 0
Specific times = TimeSpan Array
Virtual memory Limit = 0
Should I change anything.. We have a monitoring program that monitors the VM and ram available on the machine We have 6 gigs available and 5 gigs goes to the w3wp.exe. Thanks again...
Disable Overlapped Recycle = False
Disable Recycling for Configuration Changes = False
Private memory limit = 0
Regular time interval 1740
Request limit 0
Specific times = TimeSpan Array
Virtual memory Limit = 0
Should I change anything.. We have a monitoring program that monitors the VM and ram available on the machine We have 6 gigs available and 5 gigs goes to the w3wp.exe. Thanks again...
By the values on your App Pool, you are already setup to recycle ever 1740 minutes ( 29 hours ). How much traffic does your website get? Is it dynamic? What does it do? It sounds like your Application might be using page caching to increase the performance, but with an infinite number of QueryString combinations this can quickly consume your memory.
I have a Colocated Server running Server 2008 R2 with 8GB of RAM. Its running about 20 Applications. My memory usage per application ( W3Wp.exe ) averages between 70 MB up to 150 MB. My memory usage usually runs between 3 GB - 3.5GB. This is mainly because I have allocated 1.2 GB just for MYSQL ( to keep things fast ). 5 GB for a single application to me means there is something wrong. Of course this can depend on what your application's purpose is, but for the average application this wouldn't be right.
I have a Colocated Server running Server 2008 R2 with 8GB of RAM. Its running about 20 Applications. My memory usage per application ( W3Wp.exe ) averages between 70 MB up to 150 MB. My memory usage usually runs between 3 GB - 3.5GB. This is mainly because I have allocated 1.2 GB just for MYSQL ( to keep things fast ). 5 GB for a single application to me means there is something wrong. Of course this can depend on what your application's purpose is, but for the average application this wouldn't be right.
Kevin,
There is no way to determine what the expected load of your server is. Every server has different hardware specs, and every application is written differently. However to use 100% CPU all the time is not the kind of performance you should be getting. In the future I may write another blog that focuses on this topic specifically, but for the mean time here are my suggestions.
1. You have a never ending logic loop. Like a While Loop that never terminates. However this is unlikely as it would bring your server down rather quickly.
2. You are running your Application in Debug=true mode. When an application is running in debug mode this creates additional overhead. However even if you are running in debug mode, this usually doesn't cause 100% CPU usage.
3. SQL / MySQL databases need to be indexed. This is a common mistake with new developers. They setup their tables then they don't create the indexes for each table. You should make sure you have an index created to full full every WHERE statement used in your queries. Without this MySQL can quickly consumer all your CPU with queries. Though if this was the case you would see the MySQL process using up all the CPU and not your W3WP.exe.
4. Sessions are being held for too long. This is more of a memory issue than a CPU one. In most cases there is no reason to hold your sessions for longer than 20 minutes ( default ). If you set this higher, once a user leaves your website all the viewstate info from the pages they viewed will be kept in memory in case they return.
5. Disable Viewstate on Label controls. Handling the ViewState can cause additional overhead on the CPU. Disabling this on Label controls that most of the time do not need their states preserved on postback will speed up your application.
6. Bad coding practices in your application. Example for long strings use StringBuilder instead of a string object.
7. Increase your Query Cache on MySQL, or create a custom object cache form within your application to cut down on Database queries.
There is no way to determine what the expected load of your server is. Every server has different hardware specs, and every application is written differently. However to use 100% CPU all the time is not the kind of performance you should be getting. In the future I may write another blog that focuses on this topic specifically, but for the mean time here are my suggestions.
1. You have a never ending logic loop. Like a While Loop that never terminates. However this is unlikely as it would bring your server down rather quickly.
2. You are running your Application in Debug=true mode. When an application is running in debug mode this creates additional overhead. However even if you are running in debug mode, this usually doesn't cause 100% CPU usage.
3. SQL / MySQL databases need to be indexed. This is a common mistake with new developers. They setup their tables then they don't create the indexes for each table. You should make sure you have an index created to full full every WHERE statement used in your queries. Without this MySQL can quickly consumer all your CPU with queries. Though if this was the case you would see the MySQL process using up all the CPU and not your W3WP.exe.
4. Sessions are being held for too long. This is more of a memory issue than a CPU one. In most cases there is no reason to hold your sessions for longer than 20 minutes ( default ). If you set this higher, once a user leaves your website all the viewstate info from the pages they viewed will be kept in memory in case they return.
5. Disable Viewstate on Label controls. Handling the ViewState can cause additional overhead on the CPU. Disabling this on Label controls that most of the time do not need their states preserved on postback will speed up your application.
6. Bad coding practices in your application. Example for long strings use StringBuilder instead of a string object.
7. Increase your Query Cache on MySQL, or create a custom object cache form within your application to cut down on Database queries.
Thank you for this helpful article. I have just moved to a VPS as I needed some additional control over trust states and within a month am finding that he memory on the server is causing us continual headaches. Your artlcle has at least given me some items to check to help reduce unneccessary load.
Everyone, I have this article with a new section on Session Timeout. I have found that reducing your Session Timeouts can help with reducing the size of your W3WP.exe. Just make sure your timeout is long enough so people can use your submit forms.
MAINMENU
CATEGORIES
AUTHORS