# ColdFusion 10 update 5 - security update - now available

November 19, 2012 / Rakshith Naresh

Administrator | Adobe ColdFusion 10 | Announcements | Application Server | productivity | Rapid Application Development | Tomcat | web application security

The ColdFusion 10 Update 5 is now available for install within your administrator. Update 5 is a security update that resolves a vulnerability affecting ColdFusion on Windows Internet Information Services (IIS), which could result in a Denial of Service condition. Adobe recommends users update their product installation.

Refer the security bulletin for all the details associated.

### 52 comments so far ↓

• 1 Adam Cameron // Nov 19, 2012 at 9:54 AM
My experiences thusfar:

Note the title: "ColdFusion 10 Update 5: mostly smooth"

Good work. Let's see how other people go...
• 2 Steve W // Nov 19, 2012 at 10:10 AM
Is running C:\ColdFusion10\cfusion\runtime\bin\wsconfig.exe -upgrade sufficient or do users need to uninstall and re-install the connector?
• 3 Subodh // Nov 19, 2012 at 12:43 PM
Subscribing
• 4 charlie arehart // Nov 19, 2012 at 12:47 PM
The update worked for me. Requesting the URL that would cause failure before (kill the application pool) no longer does after the update.

And since this update 5 addresses issues related specifically to the IIS Connector for CF10, readers of this will also want to note the other new blog entry here created a couple of days ago, "Tuning ColdFusion 10 IIS Connector configuration":

http://blogs.coldfusion.com/post.cfm/tuning-coldfusion-10-iis-connector-configuration
• 5 Liam // Nov 19, 2012 at 12:54 PM
Subscribing
• 6 Tim // Nov 19, 2012 at 12:59 PM
Subscribing
• 7 Ron Stewart // Nov 19, 2012 at 7:02 PM
Subscribing...
• 8 Ron Stewart // Nov 19, 2012 at 7:41 PM
I updated one of my development boxes, running CF10 in standalone mode (just Tomcat on port 8500). The update downloaded and appeared to stop the server fine, but it never restarted. I have manually stopped and restarted the server on the command line twice and it is completely non-responsive. Not serving static content, CFML, or the CF admin. The log in cfusion/hf-updates/hf-10-00005/ shows nothing but successes.

Any thoughts on where to start trouble-shooting would be greatly appreciated.
• 9 Ron Stewart // Nov 19, 2012 at 8:12 PM
Looking in ../cfusion/logs/coldfusion-error.log, I see lumps that look like the following for each attempt to start ColdFusion after applying HF5, and these same lumps do not exist prior to applying HF5:

Nov 19, 2012 9:00:57 PM org.apache.catalina.core.AprLifecycleListener init
INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: .:/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java
Nov 19, 2012 9:00:58 PM org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["http-bio-8500"]
Nov 19, 2012 9:00:58 PM org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["ajp-bio-8012"]
Nov 19, 2012 9:00:58 PM org.apache.catalina.core.StandardService startInternal
INFO: Starting service Catalina
Nov 19, 2012 9:00:58 PM org.apache.catalina.core.StandardEngine startInternal
INFO: Starting Servlet Engine: Apache Tomcat/7.0.23
javax.servlet.ServletException: The configuration file cound not be found at /WEB-INF/cfform/flex-config.xml
at flex.server.j2ee.cache.CacheFilter.setupFlexService(CacheFilter.java:93)
at flex.server.j2ee.cache.CacheFilter.init(CacheFilter.java:76)
at coldfusion.bootstrap.BootstrapFilter.init(BootstrapFilter.java:34)
at org.apache.catalina.core.ApplicationFilterConfig.initFilter(ApplicationFilterConfig.java:277)
at org.apache.catalina.core.ApplicationFilterConfig.getFilter(ApplicationFilterConfig.java:258)
at org.apache.catalina.core.ApplicationFilterConfig.setFilterDef(ApplicationFilterConfig.java:382)
at org.apache.catalina.core.ApplicationFilterConfig.<init>(ApplicationFilterConfig.java:103)
at org.apache.catalina.core.StandardContext.filterStart(StandardContext.java:4624)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5270)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1525) at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1515)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:680) Nov 19, 2012 9:00:59 PM org.apache.catalina.core.ApplicationContext log INFO: failed to load: flex.server.j2ee.cache.CacheFilter Nov 19, 2012 9:00:59 PM org.apache.catalina.core.StandardContext filterStart SEVERE: Exception starting filter CFCacheFilter javax.servlet.ServletException: javax.servlet.ServletException: The configuration file cound not be found at /WEB-INF/cfform/flex-config.xml at coldfusion.bootstrap.ClassloaderHelper.initFilterClass(ClassloaderHelper.java:159) at coldfusion.bootstrap.BootstrapFilter.init(BootstrapFilter.java:34) at org.apache.catalina.core.ApplicationFilterConfig.initFilter(ApplicationFilterConfig.java:277) at org.apache.catalina.core.ApplicationFilterConfig.getFilter(ApplicationFilterConfig.java:258) at org.apache.catalina.core.ApplicationFilterConfig.setFilterDef(ApplicationFilterConfig.java:382) at org.apache.catalina.core.ApplicationFilterConfig.<init>(ApplicationFilterConfig.java:103) at org.apache.catalina.core.StandardContext.filterStart(StandardContext.java:4624) at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5270) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150) at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1525)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1515) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
Caused by: javax.servlet.ServletException: The configuration file cound not be found at /WEB-INF/cfform/flex-config.xml
at flex.server.j2ee.cache.CacheFilter.setupFlexService(CacheFilter.java:93)
at flex.server.j2ee.cache.CacheFilter.init(CacheFilter.java:76)
... 15 more
Nov 19, 2012 9:00:59 PM org.apache.catalina.core.StandardContext startInternal
SEVERE: Error filterStart
Nov 19, 2012 9:00:59 PM org.apache.catalina.core.StandardContext startInternal
SEVERE: Context [/] startup failed due to previous errors
SEVERE: The web application [/] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@5f78dc08]) and a value of type [flex.util.ServletPathResolver] (value [flex.util.ServletPathResolver@77b5c22f]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
Nov 19, 2012 9:00:59 PM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["http-bio-8500"]
Nov 19, 2012 9:00:59 PM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["ajp-bio-8012"]
Nov 19, 2012 9:00:59 PM com.adobe.coldfusion.launcher.Launcher run
INFO: Server startup in 3142 ms
• 10 Ron Stewart // Nov 19, 2012 at 8:35 PM
I should be more specific: the lumps related to being unable to find /WEB-INF/cfform/flex-config.xml and problems starting the CFCacheFilter (?) are the pieces that have appeared after applying HF5.
• 11 charlie arehart // Nov 19, 2012 at 8:56 PM
That's really odd, Ron. I've helped a lot of people with such challenges, but that's not one I've seen before.

I'll note that that file (within my CF10 install) is not only there but hasn't changed since 2004. Can you confirm first if it's there (C:\ColdFusion10\cfusion\wwwroot\WEB-INF\cfform)?

Assuming it is, then I might wonder if perhaps instead the problem is that some configuration entry that points to the file--or to the web-inf or perhaps even the [cf10]\cfusion\wwwroot\ path entirely, is what's gotten changed somehow.

Hopefully someone from Adobe (or else) would perhaps connect some dots better for you.
• 12 Ron Stewart // Nov 20, 2012 at 4:27 AM
@Charlie: Thanks for the follow up. The file is indeed present at /Applications/ColdFusion10/cfusion/wwwroot/WEB-INF/cfform/flex-config.xml, with a file date of Nov 24, 2004 and a file size of 24453b. Permissions on the file are identical to the other files in that folder. As I noted, my install was fine prior to applying HF5, but fails to start after. As a bit of background, this box is running Mac OS X 10.6.8 and Java 1.6.0_37, and has worked well with CF10 to this point. I run CF9 behind Apache, with CF10 and Tomcat in standalone mode, on this box and have been using to to move some of our apps to CF10. If it is one of the configuration files, it would have had to have been changed as part of applying HF5 and I am kicking myself for not backing up the various Tomcat configuration files prior to applying the hotfix. I guess I've gotten lazy in that regard where I've not had problems with any of the prior hotfixes. Hopefully, someone from Adobe can shed some light on this...
• 13 Ron Stewart // Nov 20, 2012 at 4:27 AM
@Charlie: Thanks for the follow up. The file is indeed present at /Applications/ColdFusion10/cfusion/wwwroot/WEB-INF/cfform/flex-config.xml, with a file date of Nov 24, 2004 and a file size of 24453b. Permissions on the file are identical to the other files in that folder. As I noted, my install was fine prior to applying HF5, but fails to start after. As a bit of background, this box is running Mac OS X 10.6.8 and Java 1.6.0_37, and has worked well with CF10 to this point. I run CF9 behind Apache, with CF10 and Tomcat in standalone mode, on this box and have been using to to move some of our apps to CF10. If it is one of the configuration files, it would have had to have been changed as part of applying HF5 and I am kicking myself for not backing up the various Tomcat configuration files prior to applying the hotfix. I guess I've gotten lazy in that regard where I've not had problems with any of the prior hotfixes. Hopefully, someone from Adobe can shed some light on this...
• 14 Ron Stewart // Nov 20, 2012 at 6:27 AM
@Charlie: the only reference I can find to the CFCacheFilter in any of the configuration files is in cfusion/wwwroot/WEB-INF/web.xml. I have access to a second box configured similarly but still on HF4. I've done a diff between the web.xml file from the non-functional HF5 box and the functioning HF4 box, and the two web.xml files are identical with the exception that I've added an additional welcome-file in the welcome-file-list on the HF4 box. Based on that, it would seem to be something much more fundamental with the hotfix than just a change to the configuration file.
• 15 charlie arehart // Nov 20, 2012 at 8:29 AM
@Ron, yep it would seem so. Though I'd just personally be less inclined to think it's necessarily "something wrong with the hotfix" (which would affect everyone) and instead maybe something about your environment which may either be unique, or may be less common than most, since we've (as a community) not yet heard others reporting it.

In fact, you're saying that the instance still won't start at all, right? I'd think we'd hear a lot of screaming if it was happening to others.

But still, I appreciate it's very important to you. I'm still inclined to think there's some configuration matter that's somehow been affected (and I don't think it's necessarily the cache-filter specifically. It could be something else, for which that is more a secondary problem than the primary one.)

Since you DO have that other CF10 implementation on the other box, just at Updater 4, how about doing a full directory compare. If you don't have a tool to facilitate that, I'll note that BeyondCompare is wonderful on Windows, but I don't know what's great on OSX. Maybe you'll say something's built-in. I will note that I keep a list of such file/directory comparison tools as a part of my CF411.com site of tools and resources, specifically http://www.cf411.com/filecomp.

Let's see if you may spot something. It could be literally just a single missing closing bracket or something in an XML file, or it may be something more (like some files missing). It could in the end be some sort of permission thing. We'll have to wait in see.

Of course, these are just my ideas. Someone from Adobe or another reader may have better thoughts for you.
• 16 Ron Stewart // Nov 20, 2012 at 12:58 PM
@Charlie: Interesting idea. I pulled a copy of the cfusion folder off of the HF5 box and used diffmerge to compare that to the working HF4 box. 5532 identical files, 306 different, 330 without peers (e.g., exist one one side but not the other), 669 folders. If I throw out the differences between the hf-updates, logs, and wwwroot/WEB-INF/cfclasses folders, there are very few differences... after spending an hour looking at the differences between the files I do see there, I'm coming up empty as to any of them that would be causing the breakage. The differences are things related to the fact that I am "ron" on one box and "stewrp" on the other, a couple of differences in paths, differences in how CF itself is configured on the two servers (e.g., differences in the various neo-*.xml files). Nothing there jumps out as a potential cause for the breakage.
• 17 charlie arehart // Nov 20, 2012 at 5:09 PM
@Ron, wow, bummer. But it would seem there just has to be some difference somewhere, right? Let's hope someone from Adobe may jump in at some point.
• 18 Rupesh // Nov 21, 2012 at 3:06 AM
@Ron, the only change in update 5 is in the connector and there has been absolutely no other change. So I am not able to understand why it would throw this error.
Can you share with us how you applied the update? Was it through the admin or was it through the command line? Hotfix installer always creates a backup and therefore it is always possible to uninstall an update and go back to the previous state. Does uninstalling update 5 solve this problem?

I will also ask the engineers to look into this.
• 19 Ron Stewart // Nov 21, 2012 at 5:12 AM

Again, thanks for the follow-up and let me know how I can help...
• 20 Krishna // Nov 22, 2012 at 2:53 AM
@Ron:
Can you please send all the .log files that are under
2. And also please send us the following logs that are under <ColdFusion10>\cfusion\logs\
coldfusion-out.log,coldfusion-err.log,exception.log

Thanks,
Krishna
• 21 Krishna // Nov 22, 2012 at 3:58 AM
@Ron:
Can you please send all the .log files that are under
2. And also please send us the following logs that are under <ColdFusion10>\cfusion\logs\
coldfusion-out.log,coldfusion-err.log,exception.log

Thanks,
Krishna
• 22 Ron Stewart // Nov 22, 2012 at 8:22 AM
@Krishna: As I noted above in my response to Rupesh, I am on holiday until this weekend. I will send the requested log files late Saturday or early Sunday when I return. Thanks for the follow-up.
• 23 Aaron Neff // Nov 26, 2012 at 12:06 AM
I've been running Updater 5 fine for a week now.

Verified the vulnerability is gone. Good.

Reviewed CF log files from May thru now. Found 0 errors introduced by Updater 5. And all previously reported errors are fixed. Good.

As far as Updater 5 goes, very good!

FWIW: On _one_ machine, installing updates via Updates page of CF Admin *always* fails. After a few seconds, the right frame flashes white for a second - but CF never stops and updater never starts. Had same issue on same machine w/ earlier updates. Installing updates via command line always works fine tho.

Thanks,
-Aaron
• 24 Christian // Nov 26, 2012 at 7:51 AM
@Rupesh
If only the connector was changed in Update 5, then why does the change log show that many others files were modified?

Modified:   C:\ColdFusion10\cfusion\bin\cf-startup.jar
Modified:   C:\ColdFusion10\cfusion\bin\cfcompile.bat
Modified:   C:\ColdFusion10\cfusion\bin\cfcompile.sh
Modified:   C:\ColdFusion10\cfusion\bin\cfinfo.bat
Modified:   C:\ColdFusion10\cfusion\bin\cfinfo.sh
Modified:   C:\ColdFusion10\cfusion\bin\coldfusion.exe
Modified:   C:\ColdFusion10\cfusion\bin\coldfusionsvc.exe
Modified:   C:\ColdFusion10\cfusion\lib\ib6core.jar
Modified:   C:\ColdFusion10\cfusion\lib\ib6http.jar
Modified:   C:\ColdFusion10\cfusion\lib\ib6swing.jar
Modified:   C:\ColdFusion10\cfusion\lib\ib6util.jar
Modified:   C:\ColdFusion10\cfusion\runtime\lib\tomcat-coyote.jar
Modified:   C:\ColdFusion10\cfusion\runtime\lib\wsconfig.jar
Modified:   C:\ColdFusion10\cfusion\wwwroot\CFIDE\scripts\ajax\package\cfwebsocketCore.js
• 25 Aaron Neff // Nov 26, 2012 at 1:00 PM
@Christian, updates are comprehensive as they include the fixes from prior updates. I believe Update 5 is essentially just a fixed Update 3 and includes Updates 1 and 2.

Thanks,
-Aaron
• 26 Aaron Neff // Nov 26, 2012 at 1:10 PM
Actually, the connector issue that Update 5 fixes was introduced in Update 1 (as mentioned in the security bulletin). But main point is that updates are comprehensive.
• 27 Christian // Nov 26, 2012 at 1:28 PM
@Aaron, That makes sense.

Thanks,
Christian
• 28 Aaron Neff // Nov 26, 2012 at 3:50 PM
@Christian, very cool and you're welcome. Additionally, I should've written "cumulative" (not "comprehensive"), but that word eluded me as I was out-and-about when writing those msgs. Anyhow, about the same difference =P
• 29 Rupesh // Nov 26, 2012 at 9:35 PM
@Christian, Aaron is absolutely right. The updates are cumulative and that is why you see those modified files.
• 30 Jason L // Nov 27, 2012 at 2:36 PM
So really now more issues now? Finally CF10 is stable?
• 31 pkucera // Nov 28, 2012 at 11:48 AM
The blogs mention command line vs administrator installation of updates. I have performed manual command line updates, but I am unable to locate the updates for download from anywhere. The only update that i can find to download for command line install has been the mandatory CF10 update for version control. Can someone point me to the update 5 download?
Regards,
Pete
• 32 charlie arehart // Nov 28, 2012 at 12:14 PM
@pkucera, you download the updates from within the CF Admin (even if you will apply them using the command line), in the new "Server Update" page at the bottom of the nav bar on the left.

BTW, you filled in n/a for your website on the comment form. No need to put anything at all if you don't want to.
• 33 pkucera // Nov 28, 2012 at 2:16 PM
@charlie, The problem is that for security reasons my server has no outbound service to download which makes it hard to use the CF Admin (I guess I did not specifically provide that information). Is using the CF Admin the only way to patch CF10? If so, why provide the command line option?
-Pete
• 34 charlie arehart // Nov 28, 2012 at 6:02 PM
Pete, there's a solution for you. Before sharing it, let me take a moment to share (for you and all readers) something that I think a lot of people have missed.

These blog entries about each new updater do not provide each time all that one may need to know about the new CF10 updater mechanism. They mainly just announce the newest one, and with the least that most need to know for how to apply them.

There are plenty of scenarios that need much more detail. Fortunately, there are a couple of resources from Adobe that do provide more.

Second, far more details are offered in a great blog entry from Adobe engineer Krishna, with nearly 50 questions/answers about the CF 10 autohotfix mechanism: http://www.krishnap.com/2012/09/coldfusion-10-hotfix-update-installer.html. It was written after the release of CF10, and documents some things that are new or have been learned since then.

For your specific challenge, see the question there, "What can be done if the ColdFusion server is behind the firewall and can't access the Adobe's Update site URL?", which shows how you can download the fix from another machine, and put it in place to run on the CF server (whether from the Admin or the command prompt.)

There is also info there for those behind a proxy, who would need to configure a proxy host, port, user, and/or password.

Hope that helps.
• 35 charlie arehart // Nov 28, 2012 at 6:07 PM
I could have been a little more clear in my next to last paragraph about solving your specific challenge. By saying the FAQ "shows how you can download the fix from another machine", I meant that it offers the URL to get the hotfix in your browser (in other words, you don't need to have installed CF on another machine to get it, though of course that would work also.)

• 36 Al Serize // Dec 3, 2012 at 4:15 PM
Installation went well - HOWEVER - after the installation the only .cfm page that would work is the index.cfm any other .cfm extention will not work. Has anyone else experience this'?
• 37 Aaron Neff // Dec 3, 2012 at 5:58 PM
@pkucera, I basically do what Charlie hints at (whenever CF Admin's Updates page fails to install an update):

('005' indicating Update 5)

@Al, After installation did you run wsconfig.exe to unconfigure/reconfigure the web server connector?

Thanks,
-Aaron
• 38 Al // Dec 4, 2012 at 7:05 AM
@Aaron Neff - Thanks for the reply, I did actually several times. I resulted in removing all the conectors, and files in the WSCONFIG folder and reinstalling all the conectors. Nothing seems to work. What's very interesting is that I have several sites on the same server, some of the sites do have all the .cfm pages working but some of them only the index.cfm will work when referred to as the directory, in other words www.url.com/dir/ if I try www.url.com/dir/index.cfm I get a 404 just like www.url.com/dir/anything-else.cfm

I've never seen this with CFML9.
• 39 AL // Dec 4, 2012 at 7:49 AM
After a some more research I was able to find out what was generating the issue. The binding for Jakarta's Virtual Directory was not pointing to the correct folder in the applicationHost.Config file.

I was able to locate the applicationHost.config fixed the virtualDirectory physicalPath to point to the correct wsconfig connector. Restarted IIS and it started working. There is also a way to do this withing the IIS interphase for those who prefer not to read the XML file.
• 40 charlie arehart // Dec 4, 2012 at 10:13 AM
Good to hear you solved it, Al. But I will wonder out loud (for you and other readers) if that problem may not have originated in your having run the web server connector tool while NOT having noticed the need to "run as administrator" (for those on Windows 2008 or 7, where their new tighter security makes this an issue.)

I know I keep harping on this need to "run as administrator" wherever the conversation turns to it, but I'm just saying I've seen it be THE solution for problems for some.

Sadly, as in a case like this, it may be that if it's done AFTER having used the wsconfig without having run as admin, it may be that the tool doesn't know enough about how to clean up problems caused in the previous (non-Admin) execution.

Tough situation for all of us, if this is the case. It will require such digging around to solve things (and Adobe will be blamed for the connector "not working".)

I would argue strongly that the wsconfig should be modified to detect and warn the user (on such Windows OSs) if they are NOT "running as administrator". (Some may have noticed that the mandatory update does this also--and they have noticed also that it says that even when you "are" technically "an administrator". The issue again is that you MUST use the "run as administrator" option.)

As more and more people fail to do that (if indeed they even think to re-run the web connector after running the CF10 updater), they will only continue to make the accusation that the problem is "with CF10". It's just not always "its fault". But perhaps it could do more to protect people from this Windows problem.
• 41 Ron Stewart // Dec 5, 2012 at 6:49 AM
A bit of follow-up regarding the problem I noted in comments #8 and #9, above: Krishna of the Adobe CF team provided me with a work-around for this problem and indicated that it seems to be a random bug they have encountered a couple of times and not specifically associated with HF5 (I'm just lucky, I guess). I have logged it as bug #3378447 with Adobe.

In case some poor soul happens to share my luck and encounter this same behavior, the work-around provided by Krishna is to copy the cfusion context's WEB-INF folder to the root of the drive and start the server. Server runs fine. Stop the server. Delete the copy of the WEB-INF folder at the root of the drive. Start the server. Runs fine. Go figure.

Troubling, but at least I have a working dev box again.
• 42 Dan Davis // Feb 12, 2013 at 1:57 PM
I've recently been experiencing an increase in the number of problems our CF10 Ent install has been exhibiting. It wasn't until last week that I was hit with the 503 Unavailable response and several second latency in delivering content. A restart of CF "ALWAYS" resolves the problem.

I'm curious if http://blogs.coldfusion.com/post.cfm/tuning-coldfusion-10-iis-connector-configuration will solve my problems or not. I'll paste a few of the errors that are causing me concern with respect to the connector and provide a link to the entire file below. Before you ask, the answers are yes, yes, & yes. I strictly adhere to the rules for applying the hotfixes as Charlie continues to point out to people in this thread and many others.

These log entries tend to happen just prior to a complete shutdown and 503. Sometimes things "appear" to recover, but usually they do not. CF doesn't crash or stop, Tomcat just seems to stop responding. Because we're focusing on the isapi connector, I thought to take a look at the isapi_rediect.log, and WHOA. (blb.org/downloads/isapi_redirect.log.txt)

[info] ajp_process_callback::jk_ajp_common.c (2058): current reuse count is 6 of max reuse connection 250 and total endpoint count 250
[warn] ajp_process_callback::jk_ajp_common.c (2035): AJP13 protocol: Reuse is set to false
[info] ajp_connection_tcp_get_message::jk_ajp_common.c (1313): (BLB_Live) can't receive the response header message from tomcat, network problems or tomcat (127.0.0.1:8013) is down (errno=54)
[error] ajp_get_reply::jk_ajp_common.c (2182): (BLB_Live) Tomcat is down or refused connection. No response has been sent to the client (yet)
[info] ajp_service::jk_ajp_common.c (2684): (BLB_Live) sending request to tomcat failed (recoverable), (attempt=1)
[info] jk_open_socket::jk_connect.c (626): connect to 127.0.0.1:8013 failed (errno=61)
[info] ajp_connect_to_endpoint::jk_ajp_common.c (1047): Failed opening socket to (127.0.0.1:8013) (errno=61)
[error] ajp_send_request::jk_ajp_common.c (1669): (BLB_Live) connecting to backend failed. Tomcat is probably not started or is listening on the wrong port (errno=61)
[info] ajp_service::jk_ajp_common.c (2684): (BLB_Live) sending request to tomcat failed (recoverable), because of error during request sending (attempt=2)
[error] ajp_service::jk_ajp_common.c (2703): (BLB_Live) connecting to tomcat failed.
[error] HttpExtensionProc::jk_isapi_plugin.c (2293): service() failed with http error 503

I'm working on applying the IIS connector config mention in the thread mentioned above, hoping it can solve my problems.

/Dan
• 43 charlie arehart // Feb 12, 2013 at 6:35 PM
Dan, I know you want to shortcut our asking if you did x, y, or z, but by saying simply "yes, yes, and yes", you leave us no choice. :-)

While you say you have followed "the rules for applying the hotfixes", but there are so many potential rules and gotchas that we have to ask what you may or may not have done.

So more specifically, did you remember to run the web server configuration tool after applying the hot fix? This one (Update 5) requires it. Are you saying this is the one you did? Or might you have done update 6 or 7? While those do not require it, if you (or anyone reading this) is applying this (or the later CF10 updates) as your first one, do note that this one and all before it are included (they are cumulative). This one, and updates 4 and 3, required the web server reconfiguriiton. Many miss that.

So are you saying you DID do that? And that you used the "run as administrator" option in launching the web server config tool?
• 44 Kiran // Feb 12, 2013 at 8:46 PM
Hi Dan

Can you please give more insight on the issue.

2. Have you re-configured connector after installing the hotfix?

3. Have you checked the server is reachable with 8500 (internal web server), when issue occured

4. Have you configured connector with All sites / Individial sites.

5. have you followed the steps to tweak the server from blog
(http://blogs.coldfusion.com/post.cfm/tuning-coldfusion-10-iis-connector-configuration )

6. Can you give please log a bug with steps to repro.
• 45 Kiran // Feb 12, 2013 at 8:51 PM
Hi Dan

If you can also send us the thread dump that will help us in investigating the issue further.

• 46 Dan Davis // Feb 13, 2013 at 9:57 AM
@charlie,
I'm at update 7 as of last week. Because this one was cumulative, and the fact that it stated to reconfigure the connectors, I certainly did. Being Server 2008 R2, one has to build a habit of running installs and updates like this "As Administrator". It has become common place around here.

@Kiran,
1)
Our server is dual socket E5-2640 hex-core Xeon and 32GB of memory. I have this particular CF instance configured to use 8192MB for the JVM heap. It averages about 2GB of memory usage and <10% CPU during it's "heavy load". We receive about 800k - 900k page view per day. So, as to under load, it's pretty busy all day long. CF 8 & 9 seemed to handle this load without issue, so I doubt we're stressing Cf, though those were JRun and this is Tomcat.

2 & 3)
I did reconfigure the connector. My isapi_redirect.dll claims it is version 1.2.32 dated 11/18/2012. I'll try reaching 8500 the next time it burps.

4)
I am using individual and not 'All'

5)
I did follow the tweaks from said blog post. so far things are running, but I am seeing some interesting log entries. these aren't happening regularly, but sporadic and in clusters of one or two minutes.

[info] ajp_connection_tcp_get_message::jk_ajp_common.c (1305): (BLB_Live) can't receive the response header message from tomcat, tomcat (127.0.0.1:8013) has forced a connection close for socket 2556
[error] ajp_get_reply::jk_ajp_common.c (2212): (BLB_Live) Tomcat is down or network problems. Part of the response has already been sent to the client
[error] ajp_service::jk_ajp_common.c (2677): (BLB_Live) sending request to tomcat failed (unrecoverable), (attempt=1)
[error] HttpExtensionProc::jk_isapi_plugin.c (2293): service() failed with http error 502

I'll poke around some more the next time to burps. I don't have much time as this server/site is production and users get accustomed to and come to expect 99.999% uptime. Something interesting I just noticed, the server monitor is not updating the flash charts on the overview page. It's been a while since I've been here, maybe a few weeks, so something isn't talking to somebody.
• 47 kiran // Feb 13, 2013 at 8:44 PM
Hi Dan

As i can see the server is running under heavy load. The correct tuning of connector settings will definitely help you in resolving the issue ur facing.

Thanks
Kiran Sakhare
• 48 charlie arehart // Feb 13, 2013 at 10:13 PM
@Dan, I had to ask. :-) Even if those are now regular practice for you, they aren't for most, so we couldn't presume it.

As for the Server Monitor showing blank charts, that's really nothing new. If you also don't see the "start" buttons at the top, it's just that for some reason the SWF was not able to talk to the server. If you refresh the browser page (click the browser button to do it, as ctrl-r or f5 won't work as normal, because of Flash taking over the screen), you should find that now the start buttons appear and charts that can be populated will be.

That said, it's worth noting as well that the top 2 charts on the overview page don't populate if you don't have "start monitoring" enabled. I realize you may know that. Saying it as much for other readers, too.

Finally, while you may need to do some of that web server config tweaking as Kiran (and the entry) proposes, I'll suggest that the Server Monitor (and some key reports) as well as its alerts may help you to better pin down if there may be more at play here than just "poor configuration of the CF/web server connection", though it could well be that.

This is the sort of thing where sometimes having someone with experience to "look over your shoulders" can help. Adobe may offer that for free, but if not, there are consultants (myself included) who do that sort of thing for a living. I keep a list of such consultants (meaning, yes, me and my "competitors") at http://www.cf411.com/cfconsult/. But as you can see, many of us also like to help for free on forums and blog entries. It just isn't always the most practical/expedient solution for some.

Hope you sort out your challenges soon, one way or another.
• 49 Anit Kumar // Feb 14, 2013 at 10:16 AM
Hi Dan,

Did you get a chance to tune in the connector. Please let us know, if you are still facing issues.

Regards,
Anit Kumar
• 50 Dan Davis // Feb 14, 2013 at 11:22 AM
I did make adjustments according to http://blogs.coldfusion.com/post.cfm/tuning-coldfusion-10-iis-connector-configuration

There are only three items discussed in that thread:
Max-Reuse Connections
Connection pool size
Connection pool timeout

Here is what I have:
worker.list=BLB_Live
worker.BLB_Live.type=ajp13
worker.BLB_Live.host=localhost
worker.BLB_Live.port=8013
worker.BLB_Live.max_reuse_connections=350
worker.BLB_Live.connection_pool_size=600
worker.BLB_Live.connection_pool_timeout=60

I set Max-Reuse to 350, since this same CF instance is used for other sites.

Though I have not run into any hiccups yet myself, it appears some of our visitors are? I'm seeing sporadic events as shown below in isapi_redirect.log

[info] ajp_send_request::jk_ajp_common.c (1658): (BLB_Live) all endpoints are disconnected, detected by connect check (1), cping (0), send (0)
[info] ajp_connection_tcp_get_message::jk_ajp_common.c (1305): (BLB_Live) can't receive the response header message from tomcat, tomcat (127.0.0.1:8013) has forced a connection close for socket 2408
[error] ajp_get_reply::jk_ajp_common.c (2212): (BLB_Live) Tomcat is down or network problems. Part of the response has already been sent to the client
[error] ajp_service::jk_ajp_common.c (2677): (BLB_Live) sending request to tomcat failed (unrecoverable), (attempt=1)
[error] HttpExtensionProc::jk_isapi_plugin.c (2293): service() failed with http error 502

For the record, this one CF instance drives 7 individual web sites, but only 2 of them generate any significant traffic. The other 5 are internal and extremely low traffic. At this point, configuring the connector outside of wsconfig, the rest is groping in the dark. How does one go about measuring what the needs are for tuning Tomcat and the connector? What is considered "heavy load" and what adjustments do you make to avert problems?

@charlie
I understand and agree about "regular practice".

Something has definitely changed recently as all of this has worked in the past. Past version of CF, include CF10, up until recent weeks. Take a look at the "Last Error". The only way I get any response out of the charts is to switch between views within Server monitor, and then it only updates a couple of times and errors out. For the record, other instances aren't yet exhibiting this same problem.

/Dan

BTW: My first submission of every post here always fails with:
"Oops... The following fields are required to post a comment: Sorry, but your comment appears to be spam and could not be submitted. "
• 51 Kiran // Feb 20, 2013 at 12:38 AM
Hi Dan

You need to update the same connection pool size value at CF side. To do so follow the details below

open the server.xml from {cf-home/cfusion/runtime/conf}, add/update the maxThreads=500 and connectionTimeout="60000" to connection node containing the AJP entry. Now the AJP entry in server.xml should look like <Connector port="8012" protocol="AJP/1.3" redirectPort="8445" tomcatAuthentication="false" maxThreads="500" connectionTimeout ="60000"> </Connector>

Restart the server / IIS. Let us know if it resolves the issue.
• 52 Kiran // Feb 20, 2013 at 12:39 AM
Hi Dan