Application deployed on Network/Remote Path – Identifying Network Latency – Improve Performance

Many a time, ColdFusion application code is deployed on a network path when your ColdFusion deployments are of large-scale and mandated to use network paths.

After setting up the server for the first time, if there is any performance hit, as the first thing you would want to cross-check few things. One of the things to determine is if there is any network latency.

Though you would have got same network within your organization same as earlier, your OS version also would have changed.

Follow the steps below to see if the performance hit is due to network latency-

When the server is under moderate or full load(with at least 8-10 requests under process), take 2 or 3 thread dumps with 30 seconds interval.

It is not appropriate to take thread dump when the server has negligible load and anlyze that as there may not be any in-process requests.

If you are not sure how to take thread dump, you can simply follow the following blog.

( Taking Thread Dumps From ColdFusion Server Programmatically )

Open the thread dump file:

Under moderate or full load server conditions, if you see more than 5-8% of running ColdFusion threads containing “WinNTFileSystem” in the thread’s stack trace –> It means that there is lot of time being spent in trying to resolve the application file paths.

Following are the sample threads having WinNTFileSystem in its dump.

"ajp-bio-8014-exec-6861" Id=13898 in RUNNABLE
 java.lang.Thread.State: RUNNABLE
 prio=5 blockedtime=28963 blockedcount=6819 waitedtime=421762 waitedcount=115
    at java.io.WinNTFileSystem.getBooleanAttributes(Native Method)
    at java.io.File.isFile(File.java:876)

 

"ajp-bio-8014-exec-6861" Id=13898 in RUNNABLE (running in native)
 java.lang.Thread.State: RUNNABLE
 prio=5 blockedtime=28961 blockedcount=6814 waitedtime=421762 waitedcount=115
    at java.io.WinNTFileSystem.canonicalize0(Native Method)
    at java.io.Win32FileSystem.canonicalize(Win32FileSystem.java:414)
    at java.io.File.getCanonicalPath(File.java:618)

 

(Note: ColdFusion threads can be identified by the name starting with "ajp-" )

For Example, if there are 50 threads with thread name starting "ajp-bio-" in the thread dump, if you see WinNTFileSystem in more than 2-3 threads, it is the time you start looking at minimizing the network latency.

 

Once you know there is latency, you would want to know how much is the latency when compared to the application existing locally.

Created a very basic network latency test program to validate this.

You can take the jar from here.

And run it from command prompt as follows:

> C:ColdFusion11jrebinjava -jar <Path of NetworkPathsTest.jar> <Network or Local Directory Path >

If the network path (Ex:- \orgserverd$) is accessible only to the ColdFusion service user, open command prompt as that user ( runas /user:<cfserviceaccount domainname>cfserviceusername CMD )

 

Examples:

Path Arguments can be one or more. More Path arguments is a good measure to see the difference clearly.

C:ColdFusion11jrebinjava -jar C:ColdFusion11NetworkPathTest.jar \orgserverd$deploycfm

C:ColdFusion11jrebinjava -jar C:ColdFusion11NetworkPathTest.jar \orgserverd$deploycfm \orgserverd$deploycfmapi

Try the same paths keeping the content same on the local machine and see the time differences.

For the same paths on local and remote, the difference in time should not be exponential.

These tests are to be performed on your ColdFusion server machine.

Once you have validations and found any latencies, it is the time to call for network optimization expertise.

 

 

ColdFusion 9/10 Charting in Cluster

If you have setup cluster and tried to use charting in ColdFusion 8/9/10, sometimes you would have received “Image Expired” message instead of showing the chart.

Charting engine runs into this problem for two reasons.

1. The scripts generated by the designer use indirect method for image retrieval, meaning that the image is
produced during page processing and is placed in cache to be retrieved by the browser.

After some period of time depending upon cache settings the image is destroyed. “Image Expired” message
tells you that the image requested by the browser is not in the server cache.

To fix this issue you might want
to change cache settings(webcharts3D.xml under cfusionlib).

 

2) In some cases the reason is that one component is uesd to create the image, but another component is used to retrieve it.

In ColdFusion cluster’s case exactly this is what it happens when sticky sessions are not enabled.

So, make sure that you enable sticky session so that the a particular request comes and is servered from the same server.

After enabling sticky session make sure to restart the web server and ColdFusion server.

 

Hope this is helpful.

 

Taking Thread Dumps from ColdFusion Server Programmatically

Many times you would want to tweak the performance of the ColdFusion server or want to debug the bottlenecks that make the server unresponsive.

To analyze this, ideally you would want to have Thead dumps and Server memory snapshot(Heap Space, Eden Space, Survivor Space, Old Gen, Perm Gen).

While you can use JDK tools like jstack to get the dumps, it is tedious to install it and schedule the thread dumps.

So, programmatically thread dumps and memory snapshots are are triggered and can be configured as a scheduler task and can be triggered on-demand as well.

Download the following zip file and move it to the server where you want to take thread dumps.

 

threaddump.zip

This zip file contains two files. One is threaddump.jar file.

Place this file under: C:ColdFusion11cfusionwwwrootWEB-INFlib and restart the server for it to load.

And the other file is the cfm file takethreaddump.cfm where the call to ThreadDump class is made and the path where the dump content should be written.

By default it is dumped to the file #GetTempDirectory()#/threaddump<Day>-<Month>-<Year>.log

(Depending on the server location it translates similar to C:ColdFusion11cfusionruntimeworkCatalinalocalhosttmpthreaddump12-8-2015.log)

You can change this to any other convenient path in the cfm file.

You can call this cfm on-demand at point of time or configure a new scheduled task to schedule it at some interval.

More number of Thread dumps are more helpful for problem analysis. So, it is better to take at some interval.

On every new day, dump is rotated automatically to a new file name.

 

When should tools.jar be updated in ColdFusion Server

tools.jar contains the utilities to compile java source into class files.

While using ColdFusion Web services stubs have to be generated.

So, this utility is required for this feature to be functional.

The tools.jar is shipped by default with ColdFusion along with full
installation of ColdFusion which would be same version of the jre that
ColdFusion is shipped with.

If you are just using this default installation and the built-in jre, your
ColdFusion web services do work fine.

However, due to security bugs or platform support you would want to update
the jre version that ColdFusion runs on.

Once you do that, please make sure to copy the tools.jar file manually from
{JDK_Home}/lib to {cf_install_home}/cfusion/lib/

Only JDK contains the tools.jar file not the jre installation. You don’t
have to install JDK on the machine where ColdFusion is installed. You can just
have jre on this machine and get tools.jar from any other machine’s JDK
installation.

And also make sure that the earlier stubs are cleared from
{cf_install_home}/cfusion/stubs/ to get the newly compiled classes.

It is necessary to update tools.jar ONLY if you are upgrading the jre to a a
higher major version.

Say, if you are upgrading from 1.8 U5 to 1.8 U51 you don’t have to update
tools.jar file.

But, say   if you are upgrading from 1.7 U55 to 1.8 U51, you have to
update tools.jar file.

 

Another case where you would want to update the tools.jar is JEE
deployments.

Say, if you are using Websphere application server with IBM JDK, you have to
place the corresponding JDK’s tools.jar under 
{cf_install_home}/cfusion/lib/.

The same applies to any other application server as well.

The simple rule is that tools.jar has to be from the same major version of
Java (minor version doesn’t matter) that ColdFusion runs on.

 

This applies to any version of ColdFusion and the same applies to all platforms.

You can copy tools.jar from Windows machine to Solaris macine as well as long as the version is correct.

 

Normal
0

false
false
false

EN-US
X-NONE
X-NONE

 

Some of the factors that help in deciding the memory that application needs

Would like to explain the some of the factors that
decide how much maximum memory that an application needs.

Here are two examples. One is with cffile upload action
and the other is with cfpdf thumbnail action.


Sometimes, while uploading large files using cffile
with upload action (<CFFILE ACTION= “UPLOAD”) you would have run
into the following error

“500 – Internal server error.

There is a problem with the resource you are looking
for, and it cannot be displayed.”

 

File upload is dependent on “Maximum size of post data”
value that you can set in ColdFusion Administrator.

However, it can’t exceed the total available free
memory.

Say, if 512 MB is the Xmx memory value (jvm.config) and
350 MB is already occupied by the application, even if “Maximum size of post
data” value is set to 200 MB, it can upload files of up to ~150 MB (512 MB- 350
MB).

It can’t upload files of size 200 MB.

To fix it, Xmx should be increased which is again
subjected to the available memory in that machine.

 

Here is other example:-

Say, you are using CFPDF’s Thumbnail Action and the
thumbnails background is not generated properly.
Here the issue would be insufficient memory.

        Just to hold the decoded image data,
java application needs large memory (say 1.2 GB depending on the image pixels).

It is because of the format and high pixel nature of
the image that is being converted. Number objects created are equal to the
total number of pixels in that image.

So, high pixel imge would consume more memory.

To fix this, Xmx value in jvm.config should be at least
-> Applications memory Size + 1.2 GB (from the above example) i.e.
a minimum of 1.5 GB and can be more depending on your application.

Changing the value and restarting the server would
fix this.

Normal
0

false
false
false

EN-US
X-NONE
X-NONE

/* Style Definitions */
table.MsoNormalTable
{mso-style-name:”Table Normal”;
mso-tstyle-rowband-size:0;
mso-tstyle-colband-size:0;
mso-style-noshow:yes;
mso-style-priority:99;
mso-style-parent:””;
mso-padding-alt:0in 5.4pt 0in 5.4pt;
mso-para-margin-top:0in;
mso-para-margin-right:0in;
mso-para-margin-bottom:8.0pt;
mso-para-margin-left:0in;
line-height:107%;
mso-pagination:widow-orphan;
font-size:11.0pt;
font-family:”Calibri”,sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;}

How to enable/disable Tomcat Access logs (logging of each ColdFusion request)

Sometimes, depending on your need, you may want to enable or disable the tomcat access logs, which track every request to ColdFusion. In CF10, these logs were enabled by default. In CF11, they are disabled by default. This post shows how to enable or disable them.

This can be simply done changing a setting a in <ColdFusion_Home>cfusionruntimeconfserver.xml file.

At the end of this xml file you may find the following commented tag.

<!–

<Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs" 

               prefix="localhost_access_log." suffix=".txt"

               pattern="%h %l %u %t &quot;%r&quot; %s %b" resolveHosts="false"/>

–>

To enable the logging, uncomment this tag as follows, and restart the server for the changes to be effective.

<Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs" 

               prefix="localhost_access_log." suffix=".txt"

               pattern="%h %l %u %t &quot;%r&quot; %s %b" resolveHosts="false"/>

If access logging is enabled, the logs are stored in {ColdFusion Root}cfusionruntimelogs.

Note that you can modify the attributes, such as the log filename (via the prefix attribute) and what is logged, via the pattern attribute. For more information on these and other attributes, and on Tomcat access logging in general, see https://tomcat.apache.org/tomcat-7.0-doc/config/valve.html#Access_Logging

Be aware that this feature logs EVERY request made to ColdFusion, and so these logs can grow quite large.

To disable the Tomcat access logs, sinply comment out the tag above (using 2 dashes for XML, rather than  CFML), and restart CF for that change to take effect.

Configuring JNDI Data sources in ColdFusion

For configuring JNDI data sources, it should firstly be supported by the
application server.

Tomcat server, which is built-into ColdFusion server, has the support for
this by default.

While there is a blog Here on this how to configure, I want to explain few mistakes
that are tend to be done while configuring. 

  One common mistake generally done is:-
  When the data source is configured from the ColdFusion administrator by
providing irrelevant    username/password.
  You may tend to provide Database’s username/password. This is NOT
Database’s username/password.
  It is the application server’s username and password on which data
source is configured.
  It should be left blank when the credentials are not required by the
server.

Prepending java:comp/env/ is required for the actual JNDI name that is
registered in xml while registering the name in administrator.

It is not mandatory that you register the JNDI data source in ColdFusion administrator.

As long as the app server supports JNDI, you can invoke the data
source that is configured in the application server in cfm source code directly
either in case of standalone installation or EAR/WAR deployment.
But for consistency and preferred names that can used in ColdFusion should be
registered in ColdFusion.

This is applicable to both ColdFusion 10 and ColdFusion 11.

 

Normal
0

false
false
false

EN-US
X-NONE
X-NONE

/* Style Definitions */
table.MsoNormalTable
{mso-style-name:”Table Normal”;
mso-tstyle-rowband-size:0;
mso-tstyle-colband-size:0;
mso-style-noshow:yes;
mso-style-priority:99;
mso-style-parent:””;
mso-padding-alt:0in 5.4pt 0in 5.4pt;
mso-para-margin-top:0in;
mso-para-margin-right:0in;
mso-para-margin-bottom:8.0pt;
mso-para-margin-left:0in;
line-height:107%;
mso-pagination:widow-orphan;
font-size:11.0pt;
font-family:”Calibri”,sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;}

Setting up ColdFusion in distributed envionment

You might want to set up ColdFusion in a distributed environment where ColdFusion is running on one machine and Web server is running on a different machine.

Following are the set of steps that have to be performed to achieve this (less error-prone):

This applies to both ColdFusion 10 and ColdFusion 11.

1) Have ColdFusion server installed in a machine.

2) Next thing is to download and install VC Runtime.

             – The version of VC Runtime that you have to install depends on the version of ColdFusion.

                Say, ColdFusion 11 needs VC Runtime 2012

                 (32-bit VC Runtime for 32-bit Web server and 64-bit VC Runtime for 64-bit Web server.

                  If you are not sure, you can install both)

                 (https://www.microsoft.com/en-in/download/details.aspx?id=30679)

                and ColdFusion 10 needs VC Runtime 2010

 

3) Copy the following contents from the machine where ColdFusion is running to the machine where Web server is running at the same location.

 C:ColdFusion11jre

 C:ColdFusion11runtimelibwsconfig.jar

 C:ColdFusion11runtimeconfserver.xml

C:ColdFusion11configinstances.xml

C:ColdFusion11configcluster.xml

 

4) Open a Command prompt and run wsconfig tool

   C:ColdFusion11>jrebinjava -jar cfusionruntimelibwsconfig.jar

    It will open a configuration window where you have to provide AppServer Host as the ColdFusion Server IP.

    Configure the connector.

Distributed environment is ready for use. Send requests to the Web server's URL with cfm files under web server root and same files under ColdFusion's Web root.

Web server would redirect these to ColdFusion, which is on some other machine.

 

 

Installing ColdFusion Hotfix as non root user

When you try to run hotfix installer from command prompt, it opens up in UI mode and enforces you to run as root user. This is done to prevent some issues you could run into.

If you don’t have the access privileges for root account but you own ColdFusion server’s runtime user, and you might want to install as that user.  Silent installation is helpful in this case.

Step 1: Download the hotfix

———————————-

Download the Hotfix from ColdFusion Administrator or directly from the following URL:

https://cfdownload.adobe.com/pub/adobe/coldfusion/11/hotfix_004.jar (Change the hotfix version according to your need in this URL)

Step 2: Creating the hotfix properties

——————————————

Create hotfix properties file and fill up with the following key/values depending on your installation.
INSTANCE_LIST
value can be a list of comma separated server instances that are
created under a particular ColdFusion installation.

INSTALLER_UI=silent
USER_INSTALL_DIR=$/$opt$/$coldfusion11
INSTANCE_LIST=cfusion,cfusionprod2  
#COMMAND_INSTALL applies only to *INX based systems.
COMMAND_INSTALL=true
 
1. The
parameter COMMAND_INSTALL=true makes sure that the Standalone server is started as
ColdFusion runtime user(default is nobody) rather the root user. So,
make sure to pass this param in Unix/Mac/Solaris machines.
2. $/$ makes the installer to interpret the path properly depending on
whether it is Windows or Non-Windows platform)

For JEE installation,
USER_INSTALL_DIR value is the WAR directory containing CFIDE, WEB-INF
and META-INF under it. An example is as follows:
$/$opt$/$jboss-as-7.1.1.Final$/$standalone$/$deployments$/$cfusion.ear$/$cfusion.war

3. And the key INSTANCE_LIST  is NOT required at all for JEE.

You are ready to run the silent installation now.

 

Step 3: Execute the Installer

——————————–

Note: For JEE installation, you should stop JEE server before installing hotfix and restart the JEE server after installation. For standalone installation, this is automatically taken care by the hotfix installer.

1. First switch to the user you want to run in the command prompt so that the hotfix installation runs as this user.

2. Make sure that the files that are under the ColdFusion directory are owned by this user. In case of JEE deployment, owner could be different, please change those to this user once.

3. cd to the directory containing hotfix_004.jar that you have downloaded in your command prompt and run the following command.
>/opt/coldfusion11/jre/bin/java -jar hotfix_004.jar -f <hotfix_properties_file_path_that_you_created_above_in_step2>

 

Update released for ColdFusion Builder 3

Update 3 for ColdFusion Builder 3 is released.

This update is
primarily a companion update to ColdFusion 11 Update 3 which ahs added support for
PhoneGap 3.5.

Normal
0

false
false
false

EN-US
X-NONE
X-NONE

/* Style Definitions */
table.MsoNormalTable
{mso-style-name:”Table Normal”;
mso-tstyle-rowband-size:0;
mso-tstyle-colband-size:0;
mso-style-noshow:yes;
mso-style-priority:99;
mso-style-parent:””;
mso-padding-alt:0in 5.4pt 0in 5.4pt;
mso-para-margin-top:0in;
mso-para-margin-right:0in;
mso-para-margin-bottom:8.0pt;
mso-para-margin-left:0in;
line-height:107%;
mso-pagination:widow-orphan;
font-size:11.0pt;
font-family:”Calibri”,sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;}

This is just an Update for the ColdFusion Builder 3
installation that were installed prior to the build number 292483.
Corresponding full installers with this update embedded were already released
in December. Automatic update notification takes care of notifying you, if this
update applies to your installation or not. So, if your installation is the one
that was released in December, this update is not applicable and won’t show you
the notification. 

This update has the following 3 bug fixes.  

 

ID

TITLE

Product
Area

3768717

Remote servers details not retained on Builder restart.

 

Server Manager

3837075

Exception when generating build for mobile project

 

Mobile Support

3848665

PhoneGap version for Mobile project should be updated to
3.5

Mobile Support

Through ColdFusion Builder 3, you would get an automatic notification to apply the Update.

 

In the month of August 2014, we had
refreshed the update 2 just fixing the bugs arised out of Update 2. Ram has blogged
it as Update 3 mistakenly. It should have been blogged as Refreshed build for
Update 2. Our apologies for the confusion with regard to the update number. 

 

 

Thanks,

Krishna