WHITESITES BLOG PHOTOS PLACES POPULAR CATEGORIES

w3wp.exe using too much memory and resources

w3wp.exe using too much memory and resources

w3wp.exe using too much memory and resources

Posted on Oct 1, 2009 by Author

If you are here, the chances are you have already gone through a few pots of coffee, and lost a few hairs over trying to figure out why your W3WP.exe process is consuming so much memory.  The solution is actually very simple, but first a little background information.

What is the W3WP.exe process?


The W3WP.exe is the IIS worker process, that handles requests, maintain sessions, viewstate, and the cache.  Its been around Since IIS 6 on Windows Server 2003, and is still found as part of IIS 7.5 with Server 2008 R2.  On a properly configured, and programmed website the W3WP.exe can usually stay under 100 Megs.   If you are running a 64bit OS it could be a little larger.  The actual size of the W3WP.exe will be determined by how much traffic your website gets, and how many pages it serves.  The 100 Meg figure is just an estimate from my own experiences.  Application Pools with multiple websites can easily go over this figure, so don't use it some magic number.  But if you are running a website that has relatively low traffic ( under 20K impressions daily ), and your W3WP.exe is taking up 100+ MB of Memory, then you might want to make some changes.

Why do I care if W3WP.exe is large?


Lets say that you have a website running on a shared hosting account.  As your site grows, it will start to experience bottle necks from the limited resources it has access to on the shared box.  Remember the average shared server is running on average 300 websites.  You site is constantly fighting other sites for CPU time and resources.  Eventually you out grow the shared environment, and are forced to upgrade to either a VPS or a fully dedicated server.  But what you don't know is your site could last much longer on the shared box or that small VPS, if you were to clean up your code. 


Most popular causes of W3WP.exe using too much memory



Page Caching Dynamic Websites


Some Programmers will use Page Caching on their ASP.NET pages to improve request speeds. Even though this does work.  It will quickly eat up all your memory.  For Page Caching to work you would need to cache each page and every combination based on its querystring variables.  For Members sites this could result in millions of different pages.  A more effective way of using caching is to Cache objects that do not change, and are used throughout the website.  Things like your menu bar.  Or maybe the details for an event.  This will reduce the number of calls to the database server, and speed up your site.  Some may argue that the function of a Database server is to cache popular queries, and that there is no difference between retrieving the data from the database server and caching the objects in your web server's memory.  This is absolutely false for many reasons.  First anytime your website has to retrieve data from the database, it has to open a connection to another process (the MySQL process), even if MySQL is running on the same box as the web server there is a very small performance penalty to establish this connection.  But if you are on a Shared Hosting Account, its very likely that the hosting company has setup a dedicated MySQL server in the datacenter somewhere.  So now your web server has to reach across the datacenter to a separate server.  At this point the latency of their network becomes an issue.  This is something I have personally experienced with multiple shared hosting providers.  This is the reason I highly recommend doing everything you can to reduce the calls to the database.  Object Caching can make a huge difference on page load times, especially when you are able to cache objects that use expensive queries.  If you want to learn more about how to custom manage objects in the ASP.NET cache, I wrote a few helper functions that are easily integrated into your websites.

ViewState Enabled on Labels


The ViewState is how ASP.NET keeps track of changes that have been made to Web Controls.  ViewState is needed anytime you are going to do a postback, like for a submit form.  But a common mistake is programmers don't disable viewstate on Label controls that are often used as place holders for long concatenated HTML strings.  If you suspect your may be guilty of this all it takes is a look at the HTML source code from your web browser to confirm it.  If your HTML source looks like this

Big asp.net Viewstate
with a really big view state then you need to disable the viewstate on your label controls.  The Viewstate is stored in the W3WP.exe process.  Its easy to see how this process can consumer so much memory when you have a view state this large and on an exponential number of web pages. The solution is to change your ASP:Label Controls from this

<asp:Label ID="mainmenu" runat="server" />
to this
<asp:Label ID="mainmenu" EnableViewState="false" runat="server" />
Lets face it there is no reason you need to preserve the ViewState of a Label Control.  So go through all your pages and add the EnableViewState="false" attribute to all your Label Controls.

Long Session Time Outs

Anytime a visitor ( organic or spider ) visits your website, ASP.NET is going to maintain a session with them.  Sessions are how the server keeps track of your users and their activity on your website.  Without Sessions things like postback would not be possible.  When ASP.NET uses sessions in the traditional way, they store a Session ID in a cookie on the client's browser.  Then every time this client communicates with the server this cookie value ( Session ID ) is passed along with the request.  The Server can then identify the visitor by their Session ID and handle the requests appropriately.  Now the server has to keep track of the visitors activity.  So each time a submit form is submitted, or a web control is used, the information related to this action is stored with the viewstate in the W3WP.exe, and is tied to the Session ID.  This memory is not released until the Session Expires.  The default on ASP.NET is for Session to Expire about 20 minutes.  This means after 20 minutes of no activity from this visitor the Session would be discarded, and the memory associated with this session would be freed.  Now if you have some lengthy submit forms which could take longer than 20 minutes to fill out, then you might need to have a longer than normal session timeout.  On blog.whitesites.com I have sessions set to 45 minutes ( sometimes I take a while to write an article ) But if your submit forms are the quick kind that can be filled out within a few minutes, then 20 minutes is more than enough.  If your website doesn't have any submit forms, then it might make sense to take your session timeouts down to 5 minutes or less.  The shorter your Session Time Outs, the less memory that will be needed to track your visitors as they navigate through your website.  If you want proof.  Set your Session Timeouts to be 6 hours.  Then watch over the next 6 hours as your W3WP.exe grows to a massive size.  I can only imagine what sites like Facebook must deal with considering their visitors ( or should I call them addicts ), do daily surfing marathons.

For a website like Blog.whitesites.com, in which the only public web control is the comment box at the bottom of the page, I could probably get by with a 15 minute session timeout.  But my backend that is run from the same Application needs much longer Session timeouts.  If I wanted to get very efficient with my memory, I would create a separate application for my backend ( control panel ) with the longer timeout, allowing me to shorten the timeout on my public application. 

So in summary.  Lots of submit forms, plus lots of unique visitors, plus a long session time out will equal a very big W3WP.exe.  You might not be able to change your submit forms, or your visitors, but you can shorten your session timeouts.

Memory Leaks


Certain objects should be cleared from memory with the Dispose() Method.  These include things like BitMap Objects.  Even though the ASP.NET Garbage Collection does a good job of taking out the trash, I have found that on heavily loaded systems if you don't release Bitmaps with the Dispose Method it tends to lock the image file you were working with.  This object will sit in your W3WP.exe taking up space.  Even though memory leaks can be a problem, they are usually very minor.  ViewState is by far the biggest Memory hog.

Unclosed Database Connections


If you use helper functions to manage your Database connections, be sure that you close the connections, when finished.  Else this will waste memory on both the W3WP.exe side and the Database Server side.

Conclusion


Even though advanced programmers might feel some of this is obvious.  I found it disturbing that I was unable to find more articles online involving something so important.  Who knows, maybe if every programmer uses my advice, this could negatively affect the bottom line of hosting companies, who's customers won't be needing to get that dedicated server just yet.  If this helped you let me know.

Permalink
145902 Visitors
233892 Views

Categories associated with w3wp.exe using too much memory and resources

Discussion

Paul | Nov 13, 2009 3:07 AM
I have never used sharepoint before, though it seems that most medium to large scale companies have it running.  It doesn't really surprise me that Sharepoint is a memory hog.  From what I have seen in the development world the server-sided technologies that require the least effort from a programmer, usually have the most chub.  Most developers would aggree that Linux with apache is the most conservative on memory usage, while asp.net can start to get a little greedy especially if you don't know what you are doing.  Then you have platforms like Cold Fusion that are just plan Fatties.  I am still a fan of ASP.NET.  If you know how to properly manage your memory and objects it can be extremely lean and fast.
jesse | Mar 23, 2010 2:37 PM
I have two Windows 2008 R2 servers up and running .Net web apps. As time goes by (2 weeks), we have seen the Memory usages go up as high as 5Gigs. The Developers are really nervous with this kind of usage. How do I explain it to them that this is normal for the 64 bit platform?   This application was 32 bit and we upgraded from Windows 2003 32 bit to Windows 2008 R2 64-bit. The servers are only web servers running IIS 7.5.
Paul | Mar 23, 2010 5:21 PM
Jesse,
First I would check to make sure the memory usage is from the W3WP.EXE files.  One thing is if you are setting up the websites in the traditional way, each website / application gets its own App Pool.  I would look into how much memory each website is consuming, to try to find out which one is taking up all the memory.  going up to 5GB on a 64 Bit Server is not bad, unless you are running a little low on memory.  Someone I do with all my applications is I schedule them to recycle the App Pools every 24 hours.  For an Application to grow over 2 weeks tells me that the Application is not set to recycle after X minutes.  If you goto the advanced properties on the Application Pool you will be able to change the recycle interval.  However I would check your applications using the tips I listed above.  Also check to see if your developer has page caching enabled.  If these are dynamic sites this can quickly consume a lot of memory.  Hope this helps
Jesse | Mar 24, 2010 8:16 AM
Thank you so much for replying to my question. We only have one web application on that web server. When I go to Advanced settings under the app pool for the web site, I see many different parameters available for Recycling:

Disable Overlapped Recycle = False
Disable Recycling for Configuration Changes = False
Private memory limit = 0
Regular time interval 1740
Request limit 0
Specific times = TimeSpan Array
Virtual memory Limit = 0

Should I change anything.. We have a monitoring program that monitors the VM and ram available on the machine We have 6 gigs available and 5 gigs goes to the w3wp.exe.   Thanks again...
Paul | Mar 24, 2010 8:45 AM
By the values on your App Pool, you are already setup to recycle ever 1740 minutes ( 29 hours ).  How much traffic does your website get?  Is it dynamic?  What does it do?  It sounds like your Application might be using page caching to increase the performance, but with an infinite number of QueryString combinations this can quickly consume your memory. 

I have a Colocated Server running Server 2008 R2 with 8GB of RAM.  Its running about 20 Applications.  My memory usage per application ( W3Wp.exe ) averages between 70 MB up to 150 MB.  My memory usage usually runs between 3 GB - 3.5GB.  This is mainly because I have allocated 1.2 GB just for MYSQL ( to keep things fast ).  5 GB for a single application to me means there is something wrong.  Of course this can depend on what your application's purpose is, but for the average application this wouldn't be right.
Paul | Jun 11, 2010 2:19 PM
Kevin,
There is no way to determine what the expected load of your server is.  Every server has different hardware specs, and every application is written differently.  However to use 100% CPU all the time is not the kind of performance you should be getting.  In the future I may write another blog that focuses on this topic specifically, but for the mean time here are my suggestions.

1. You have a never ending logic loop.  Like a While Loop that never terminates.  However this is unlikely as it would bring your server down rather quickly.

2. You are running your Application in Debug=true mode.  When an application is running in debug mode this creates additional overhead.  However even if you are running in debug mode, this usually doesn't cause 100% CPU usage.

3. SQL / MySQL databases need to be indexed.  This is a common mistake with new developers.  They setup their tables then they don't create the indexes for each table.  You should make sure you have an index created to full full every WHERE statement used in your queries.  Without this MySQL can quickly consumer all your CPU with queries.  Though if this was the case you would see the MySQL process using up all the CPU and not your W3WP.exe.

4. Sessions are being held for too long.  This is more of a memory issue than a CPU one.  In most cases there is no reason to hold your sessions for longer than 20 minutes ( default ).  If you set this higher, once a user leaves your website all the viewstate info from the pages they viewed will be kept in memory in case they return. 

5.  Disable Viewstate on Label controls.  Handling the ViewState can cause additional overhead on the CPU.  Disabling this on Label controls that most of the time do not need their states preserved on postback will speed up your application.

6. Bad coding practices in your application.   Example for long strings use StringBuilder instead of a string object. 

7.  Increase your Query Cache on MySQL, or create a custom object cache form within your application to cut down on Database queries.
christina | Jul 5, 2010 11:12 AM
Thank you for this helpful article. I have just moved to a VPS as I needed some additional control over trust states and within a month am finding that he memory on the server is causing us continual headaches. Your artlcle has at least given me some items to check to help reduce unneccessary load. 
Paul | Jul 6, 2010 2:17 AM
Everyone, I have this article with a new section on Session Timeout. I have found that reducing your Session Timeouts can help with reducing the size of your W3WP.exe.  Just make sure your timeout is long enough so people can use your submit forms.
Neel | Mar 26, 2011 10:18 AM
thanks for sharing the info it does help. infact i am looking for a similar problem that one of my client is facing and that's w3wp using cache pages are high and the processing threads gets terminated. may be i can try this out and check it with them.

thanks once again for this info.
Wilson | Apr 28, 2011 1:42 PM

Thank's for shring those tips, it does help a lot, but a have a problem in a server that is configured with application pool proccess, it occasionally triggers the error: 
Process name: w3wp.exe 
Account name: NT AUTHORITYNETWORK SERVICE
 
Exception information:
    Exception type: HttpException
    Exception message: Could not load file or assembly '3i' or one of its dependencies. The process cannot access the file because it is being used by another process. (Exception from HRESULT: 0x80070020)

On this message, "3i"is my website. Is Anyone know what is happening?

Earl | Sep 22, 2011 6:03 PM
Thanks for the great information. I would like to ask you a question if I may. Our dedicated Windows 2008 r2 Exchange 2010 server shows that multiple w3wp.exe processes are consuming most of the server's 12Gb of RAM. Is the very high memory usage an expected behavior with this Microsoft program?
paul | Sep 22, 2011 7:40 PM
@Earl
I personally have no experience with exchange server, though I have heard its quite a resource hog from other Server Admins.  I use smartermail on my server for about 46 domains and 225 email accounts.  My web interface usage typically stays within 200 - 300 Megs.

High memory usage normally is the result of a high number of sessions being created.  Every time a visitor visits a website a sessions is created.  As they browse around the site, this session is maintained.  Now there is the potential that bots are hammering your exchange server web portal.  I would recommend setting up a firewall, to block all countries that your users will not be within.  IE if your users are only North american, then block all other countries from your web admin.  Also if you have an IP address that directly points to the exchange portal.  IE ( HTTP://1.2.3.4 pulls up your exchange portal ) I would recommend removing this host header in IIS.  Have subdomains point at it but not just RAW IPs. The reason is Bots incrementally will scan IPs to see what is hosted where.  Once they find something responding to an IP request, they will try every known exploit to get in.  Bots do not usually maintain sessions, so this creates a new session on every request, and quickly eats up your memory.

Hope this helps you diagnose things.
If you want the easy solution, switch to smartermail,  Thousands of companies already have.
meenu | Oct 29, 2011 11:43 PM

dear friend,

i am troubling lot yesterday because of w3wp.exe memmory usage .It reaches 100%.

my servr specification is as follows

Server 2003 std edition.SP2
intel(R) core(TM)2Duo CPU
E6750 @ 2.66 GHz
2.67 GHz, 1.99 GB of RAM
physical address extension


we have a dedicated server .here two website is running in ths server.when this exe uses 100%memory usage,both of my site was stuck.When i immediately restarted the machine everyting was ok.but i want a permanent solution.IIS i restarted.

how to avoid this memory usage permanently?

yesterday in IIS,  i rgt click the application pool ,then click refresh and  i click recycle .

whether it will solve my issues?If like that shall i do this same process weekly?

in event viewer i am getting continuous warning message like the following...hiw can i solve these...pls help me ...it is very urgent......



Event code: 3005
Event message: An unhandled exception has occurred.
Event time: 10/30/2011 8:24:05 AM
Event time (UTC): 10/30/2011 4:24:05 AM
Event ID: 739d6c42746748d69cc04f9f5ee9feaa
Event sequence: 3
Event occurrence: 1
Event detail code: 0
 
Application information:
    Application domain: /LM/W3SVC/669154516/Root-2-129644219519687500
    Trust level: Full
    Application Virtual Path: /
    Application Path: "pathname"
    Machine name: machinename
 
Process information:
    Process ID: 1524
    Process name: w3wp.exe
    Account name: NT AUTHORITYNETWORK SERVICE
 
Exception information:
    Exception type: SqlException
    Exception message: Timeout expired.  The timeout period elapsed prior to completion of the operation or the server is not responding.
 


Paul | Oct 30, 2011 11:37 AM
@meenu
First if you have two websites running.  Create separate worker proceses so each site has its own W3WP.exe.  This will help you to determine which site is actually consuming all the memory. Second 2 GB of RAM is almost nothing.  You should always have 2x the RAM you actually intend to use.  This is because anytime your RAM usage goes over 50% Windows will rev up the page file and your system will start to suffer slow downs.  The error your are getting has to do with communication errors with SQL server, most likely the system is under stress due to the high memory usage. How much memory is SQL using?  Or are you using a Shared DB server?
meenu | Oct 30, 2011 11:45 PM
we are not using shared db.
in IIS i stopped Microsoft SharePoint Administration (Stopped) and sharedapppool .Whetehr it will solve my issue?
Kannan | Dec 22, 2011 4:32 AM
Hi,
I am having few site running in server. at times i am getting system.outof memory exception that too only for a single page. also few pages taking lot of time to process the request it goes on and on and goes to blank page :(...any suggestion appreciated.

also i have done allowing seperate connection pool and indexing,setting memory to this connection pool..no luck please help in solving out this issue.
Paul | Dec 22, 2011 12:15 PM
@Kannan
Normally that error is from an infinite loop.  Check your code and make sure you don't have any while statements that can run endlessly.
Yogesh Kamra | Jul 2, 2012 4:43 AM
Hi,

facing the same problem of w3wp.exe, it shows almost  400-500 mb size. whereas server are 2 HP G6 server - Quardcore, dual processor, 16 GB RAM and configured NLB on the servers.

here utilization of the CPU shows 15-18% and same with memory (RAM).

I am using 5 MBPS bandwidth 1:1 and graph shows that we are hardly utilizing the 2 MBPS bandwidth,

Website stop responding certain times on one server.

Please help.

Thanks

vithoba Patkar | Jul 30, 2012 2:22 AM

Facing the same problem of w3wp.exe, it shows almost 400-500 mb size. Whereas server is vsphere, dual processor, 16 GB RAM 50 GB HDD (c:drive) and configured NLB on the servers.

Here utilization of the CPU shows 72-75% and same with memory (RAM).

I am using 5 MBPS bandwidth 1:1 and graph shows that we are hardly utilizing the 2 MBPS bandwidth,

Website stop responding certain times on one server.

Please help.

Thanks
Vithoba Patkar

Facing the same problem of w3wp.exe, it shows almost 400-500 mb size. Whereas server is vsphere, dual processor, 16 GB RAM 50 GB HDD (c:drive) and configured NLB on the servers.

Here utilization of the CPU shows 72-75% and same with memory (RAM).

I am using 5 MBPS bandwidth 1:1 and graph shows that we are hardly utilizing the 2 MBPS bandwidth,

Website stop responding certain times on one server.

Please help.

Thanks
Vithoba Patkar

Paul | Jul 31, 2012 1:55 AM
@Vithoba
How many websites is the server running?
What OS?
Does every website have its own application pool?
Get your memory usage under 50%. Windows hits the page file once you break 50% memory usage.  
Paul | Jul 31, 2012 2:05 AM
What processes are consuming all the memory and how much?
400 - 500 MB still leaves 15+ GB unaccounted for.
If you don't mind what website is running on the box?

Paul | Jul 31, 2012 2:21 AM
Check all pages for Large ViewState
example look at this page https://www.ieltsidpindia.com/Contact-Us.aspx?zone=North
View the HTML Source Code
Notice the huge view state at the top of the page.
You need to disable the viewstate on the label control that lists out all your locations.  Check all pages on your site for this.  Once you correct this, I think you will see your W3WP.exe get a lot smaller.

Vithoba patkar | Jul 31, 2012 4:35 AM

Hi Paul,

the issue has been resovled thanks for your help.

Thanks

Anderson Kubota | Aug 21, 2012 5:48 PM

Hi Vithoba,
Could you share what you did to solve the problem?
I am also facing a problem where we have 2 Windows 2003 32 bits server, and one of them is hiting 500-600 MB (w3wp.exe process) and the application stops answering. It is a classic ASP application that uses includes (a lot) and a COM object.
We were working on migrating the application for a Windows 2008 64 bits, since it would use much more memory. But checking your comments, you were already using a 64 bits operating system, and the application had problems when hitting 600 MB.
Please let me know what you did.

Thanks in advance.

shimon | Oct 30, 2012 3:11 AM

i have sbs 2003 problem with w3wp.exe
i want to know who i can fix it
thank you
from jersulam holy city in the world

suthan | Nov 25, 2012 7:51 PM
HI shimon,

For windows 2003 you have fix from  microsoft ...

try the below link..

http://support.microsoft.com/kb/916984

could any one share solution for window server 2008
Anderson Kubota | Nov 26, 2012 7:37 AM
As far as I understood, you will not have problems with Windows 2008, since it is already 64 bits.
For my scenario, I realized that the problem was related to excessive consumption of memory, in other words, more simultaneous users that requested more memory, and because of the 32 bits limitation for Windows 2003, few available memory.  Everything solved after changing to a 64 bits server.
codefinger | Jan 3, 2013 12:04 PM
Thanks for the reminder on checking viewstate! I just had a bingo moment after reading your article as I found a very high correlation between the size of the view state (when viewing source) and the pages which when loaded repeatedly in my load tests have been ramping up the w3wp.exe memory usage (and not releasing it).

I suspect it goes beyond use of labels though. The affected pages have other webcontrols i.e. gridviews, placeholder controls etc whereas the better performing pages have mostly been refactored to use Jquery, ajax callbacks and html templates (that I have control of).
Jyothis | Apr 21, 2013 7:18 AM
Excellent Article - gives a lot of Info on W3WP.EXE process, appreciate your work Paul
name
Email Needed to confirm comment, but not made public.
Website
 
Type Code
Security Check
 
When you Post your Comment, you'll be sent a confirmation link. Once you click this link your thoughts will be made public.. Posts that are considered spam will be deleted, Please keep your thoughts and links relavent to this Article