Entries Tagged as 'General'

ColdFusion 2016 Update 3 early access build is now available for your testing and feedback. It includes support for Windows 10.1 and IBM Websphere Application Server 9 along with approximately 180 bug fixes.

Please note that this is a test build and should not be used in a production environment.

Refer the documents given below for more details on this update:

1. What's New in this update    
2. Issues Fixed in this update
3. Installation Instructions

The build number after applying this update should be 2016.0.03.300232.

In case, you have configured local site for receiving the update notifications, then please take back up of the URL before changing it to the pre-release URL.

We will look forward to your valuable feedback and suggestions.

 

Many a time, ColdFusion application code is deployed on a network path when your ColdFusion deployments are of large-scale and mandated to use network paths.

After setting up the server for the first time, if there is any performance hit, as the first thing you would want to cross-check few things. One of the things to determine is if there is any network latency.

Though you would have got same network within your organization same as earlier, your OS version also would have changed.

 

Follow the steps below to see if the performance hit is due to network latency-

When the server is under moderate or full load(with at least 8-10 requests under process), take 2 or 3 thread dumps with 30 seconds interval.

It is not appropriate to take thread dump when the server has negligible load and anlyze that as there may not be any in-process requests.

If you are not sure how to take thread dump, you can simply follow the following blog.

( Taking Thread Dumps From ColdFusion Server Programmatically )

Open the thread dump file:

Under moderate or full load server conditions, if you see more than 5-8% of running ColdFusion threads containing “WinNTFileSystem” in the thread’s stack trace --> It means that there is lot of time being spent in trying to resolve the application file paths.

Following are the sample threads having WinNTFileSystem in its dump.

"ajp-bio-8014-exec-6861" Id=13898 in RUNNABLE
 java.lang.Thread.State: RUNNABLE
 prio=5 blockedtime=28963 blockedcount=6819 waitedtime=421762 waitedcount=115
    at java.io.WinNTFileSystem.getBooleanAttributes(Native Method)
    at java.io.File.isFile(File.java:876)


"ajp-bio-8014-exec-6861" Id=13898 in RUNNABLE (running in native)
 java.lang.Thread.State: RUNNABLE
 prio=5 blockedtime=28961 blockedcount=6814 waitedtime=421762 waitedcount=115
    at java.io.WinNTFileSystem.canonicalize0(Native Method)
    at java.io.Win32FileSystem.canonicalize(Win32FileSystem.java:414)
    at java.io.File.getCanonicalPath(File.java:618)


(Note: ColdFusion threads can be identified by the name starting with "ajp-" )

For Example, if there are 50 threads with thread name starting "ajp-bio-" in the thread dump, if you see WinNTFileSystem in more than 2-3 threads, it is the time you start looking at minimizing the network latency.

 

Once you know there is latency, you would want to know how much is the latency when compared to the application existing locally.

Created a very basic network latency test program to validate this.

You can take the jar from here.

And run it from command prompt as follows:

> C:\ColdFusion11\jre\bin\java -jar <Path of NetworkPathsTest.jar> <Network or Local Directory Path >

If the network path (Ex:- \\orgserver\d$) is accessible only to the ColdFusion service user, open command prompt as that user ( runas /user:<cfserviceaccount domainname>\cfserviceusername CMD )

Examples:

Path Arguments can be one or more. More Path arguments is a good measure to see the difference clearly.

C:\ColdFusion11\jre\bin\java -jar C:\ColdFusion11\NetworkPathTest.jar \\orgserver\d$\deploy\cfm\

C:\ColdFusion11\jre\bin\java -jar C:\ColdFusion11\NetworkPathTest.jar \\orgserver\d$\deploy\cfm\ \\orgserver\d$\deploy\cfm\api\

Try the same paths keeping the content same on the local machine and see the time differences.

For the same paths on local and remote, the difference in time should not be exponential.

These tests are to be performed on your ColdFusion server machine.

Once you have validations and found any latencies, it is the time to call for network optimization expertise.

 


Machine learning has been a buzz word in recent times. I often hear people talking about this and their profound interest to learn more and gain expertise in this field. As a consequence, many programming languages are now having machine learning libraries available in their toolbox. For example, Java has ML libraries like WEKA, C++ has Encog (Encog is available for Java as well), PyBrain for Python etc.  These libraries come in handy and obviates the need to write basic algorithms from scratch. One such tool is WEKA (http://www.cs.waikato.ac.nz/ml/weka/)-  A data mining software in Java. You can use it as a standalone application or use it's APIs in your own Java program.

 

In this article, I will show you how to use WEKA library in ColdFusion. I am going to write a very simple Java program and add wrappers to it in ColdFusion. These wrappers can then be called from ColdFusion pages. In short, I am making the WEKA library available to ColdFusion. 

 

I will be using weather dataset located at data/weather.numeric.arff  in downloaded weka package. Weka uses ARFF format for datasets. Below is the dataset

@relation weather

@attribute outlook {sunny, overcast, rainy}
@attribute temperature numeric
@attribute humidity numeric
@attribute windy {TRUE, FALSE}
@attribute play {yes, no}

@data
sunny,85,85,FALSE,no
sunny,80,90,TRUE,no
overcast,83,86,FALSE,yes
rainy,70,96,FALSE,yes
rainy,68,80,FALSE,yes
rainy,65,70,TRUE,no
overcast,64,65,TRUE,yes
sunny,72,95,FALSE,no
sunny,69,70,FALSE,yes
rainy,75,80,FALSE,yes
sunny,75,70,TRUE,yes
overcast,72,90,TRUE,yes
overcast,81,75,FALSE,yes
rainy,71,91,TRUE,no


The above dataset contains five attributes namely outlook , temperature, humidity, windy & play. outlook is a categorical variable(it can take only three values - sunny,overcast, rainy) temperature and humidity are numeric variables, windy is again a categorical variable(it can take two values - TRUE, FALSE) and play is a categorical variable(it can take two values - yes,no). play is the class variable in our problem which means we have to predict the value of this variable for given test datapoint which contains remaining four values. 

 

So the problem at hand is a classification problem. We are given a training dataset which can be used to train a classifier and then use this model to predict the label for a test datapoint. There are many classifiers which can be used to solve this problem. But I am going to use naïve Bayes to solve this problem. It's a very simple classifier based on probability and it assumes that features are independent of one another. This means that the value outlook takes is independent of the values other attributes take.  I will not explain the mathematical background of naïve Bayes but rather provide an implementation for it. To understand the theory behind naïve Bayes, check this link.

 

Here is the Java program

 

import weka.classifiers.Classifier;
import weka.classifiers.Evaluation;
import weka.classifiers.bayes.NaiveBayes;
import weka.classifiers.trees.J48;
import weka.core.Attribute;
import weka.core.Instance;
import weka.core.Instances;
import weka.core.SparseInstance;
import java.util.*;


public class Demo {

	public static Classifier naiveBayesClassifierTrainedWithCV(Instances data, int numFolds) throws Exception
	{
		Classifier cls = new NaiveBayes();
		Evaluation eval = new Evaluation(data);
		
		if(data.classAttribute().isNominal())
			data.stratify(numFolds);
		
		for(int i=0;i<numFolds;++i)
		{
			Instances train = data.trainCV(numFolds,i);
			Instances test = data.testCV(numFolds, i);
			
			cls.buildClassifier(train);
			eval.evaluateModel(cls, test);
		}
		
		return cls;
	}
	
	public static Instance generateTestSample(String overcastValue, int temp, int hum, String wind,Instances data)	{
		
		//Create nominal attribute
		List<String> outlookValues = new ArrayList<String>();
		outlookValues.add("sunny");
		outlookValues.add("overcast");
		outlookValues.add("rainy");
		Attribute outlook = new Attribute("outlook", outlookValues);
		
		//Create numeric attribute
		Attribute temperature = new Attribute("temperature");
		Attribute humidity = new Attribute("humidity");
		
		List<String> windyValues = new ArrayList<String>();
		windyValues.add("TRUE");
		windyValues.add("FALSE");
		
		Attribute windy = new Attribute("windy",windyValues);
		
		
		Instance testSample = new SparseInstance(5);
		
		testSample.setDataset(data);
		testSample.setValue(outlook, overcastValue);
		testSample.setValue(temperature, temp);
		testSample.setValue(humidity, hum);
		testSample.setValue(windy, wind);
		
		return testSample;
	}
	
	public static double[] getDistributionInstanceForATestSample(Classifier cls, Instance data) throws Exception	{
		
		return cls.distributionForInstance(data);
	}
}


Download weka.jar from http://www.cs.waikato.ac.nz/ml/weka/downloading.html. Add this jar to the classpath and compile the above program.

 

In the above program, naiveBayesClassifierTrainedWithCV is used to train a naïve Bayes classifier with the dataset that is passed as an argument. The second argument numFolds is used to mentioned the number of folds for cross-validation. For e.g., 10-fold cross-validation requires numFolds = 10. generateTestSample is used to generate a test sample for which a class label will be predicted by the classifier. getDistributionInstanceForATestSample is used to get the probability distribution for a test sample predicted by the classifier.

 

Following is a wrapper for these methods in ColdFusion. I have created a CFC for the same

component output=false
{
	property name="demo";
	property name="bufferedReference";
	property name="fileReference";
	property name="fileObj";
	property name="bufferedObj";
	property name="instanceReference";
	property name="instanceObj";
	property name="model";
	
	//This function loads ARFF file
	function Init()
	{
		demo = CreateObject("java","Demo");
		
		//Create a buffered reference
		bufferedReference = CreateObject("java","java.io.BufferedReader");
		
		//Create a file reference
		fileReference = CreateObject("java","java.io.FileReader");
		
		//Create a file object
		fileObj = fileReference.init("C:/ColdFusion11/cfusion/wwwroot/weka/weather.numeric.arff");
		
		//Create a buffered reader object
		bufferedObj = bufferedReference.init(fileObj);
			
		instanceReference = CreateObject("java","weka.core.Instances");
		instanceObj = instanceReference.init(bufferedObj);
		
		instanceObj.setClassIndex(instanceObj.numAttributes() - 1);
	
	}
	
	function naiveBayesClassifierTrainedWithCV()	{
		
		model = demo.naiveBayesClassifierTrainedWithCV(instanceObj,10);
		return model;
	}
	
	function  getDistributionInstanceForATestSample(required string outlook, required numeric temp, required numeric humidity, required string windy)	{
		
		sample = demo.generateTestSample(JavaCast("string",outlook),JavaCast("int",temp),JavaCast("int",humidity),JavaCast("string",windy),instanceObj);
		distribution = demo.getDistributionInstanceForATestSample(model,sample);
		return distribution;
	}
}

Above file is named as MachineLearning.cfc. Init() method is used to load the ARFF file. Rest of the methods are just wrappers around the corresponding Java methods. This CFC is used as follows

<cfset mlObj = CreateObject("MachineLearning")>
<cfset mlObj.Init()>
<cfset NB = mlObj.naiveBayesClassifierTrainedWithCV()>
<cfset distributionForTestSample = mlObj.getDistributionInstanceForATestSample("sunny",80,95,"TRUE")>
<cfsavecontent variable="html">
	<cfoutput>
		<html>
			<head>
				<title>WEKA Demo</title>
			</head>
			<body>
				<h3>Naive Bayes</h3>
				<p><pre>#NB#</pre></p>
				<hr />
				<h3>Classifying a sample</h3>
				<h4>t = (Outlook = "sunny", Temperature = 76, Humidity = 95, Windy = "TRUE")</h4>
				<p>
					Yes = #distributionForTestSample[1]#, No = #distributionForTestSample[2]#
				</p>
			</body>
		</html>
	</cfoutput>
</cfsavecontent>
<cfoutput>#html#</cfoutput>

MachineLearning object is created and naïve Bayes classifier is called on that object. Once the model is trained, a test sample is created and we want to know what is the label that is predicted by the classifier. Instead of getting the label, probability distribution of the test sample over the class labels is calculated i.e. probability of test sample being yes and probability of the sample being no. This is achieved by calling getDistributionInstanceForATestSample method. Below is the output that we get

As you can see, for test sample("sunny",76,95,"TRUE") the classifier predicts with 60% probability that play=yes and with 40% that play=no.

 

So this was a simple example on how to use Weka in ColdFusion. Perhaps, this idea can extended and a ML library for ColdFusion can be created.

With ColdFusion 11 Update 3, we have introduced a new parameter called "getHeaders", in the "cfExchangeMail" tag. It accepts a boolean value. When set to true, cfExchangeMail returns a query with an additional "InternetHeader" column. This column contains a struct containing key-value pairs of the email-headers associated with each message.

Email message headers provide technical details about the message, such as who sent it, the software used to compose it, the version of the MIME protocol used by the sender etc. 

On Exchange 2010, the fields that are returned are: CC, Content-Transfer-Encoding, Content-Type, Date, From, MIME-Version, Message-ID, Received, Return-Path, Subject, To, X-MS-Exchange-Organization-AuthAs, X-MS-Exchange-Organization-AuthSource, X-Mailer.

You may reference this weblink for the detailed list of the fields and their description.

You can put this new feature to any good use that suites your purpose. I will dwell on one such use case below.

In MS Exchange 2010 and later, the "ToId" column in the retrieved messages query contains the primary email address of the sender. A primary email address can have multiple aliases. If you need to retrieve the email-alias the message was sent to, you can make use of this new attribute.

Here's an example that demonstrates the usage the tag in the context of the use case discussed above:

<cfmail from="#frm_usr_email#" to="#to_usr_alias#" cc="#cc_usr_alias#" subject="#msg_sub#"  server= "#exchangeServerIP#" port = "25">

----------- testing mail to an alias address ------------

</cfmail>

<cfset sleep(5000)>

<cfexchangeConnection action="open" username ="#to_usr#" password="#password#" server="#exchangeServerIP#" serverversion="#version#" protocol="#protocol#" connection="excon">

<cfexchangemail action="get" name="usr_msgs" connection="excon" getheaders=true folder="Inbox">

<cfexchangefilter name="fromID" value='#frm_usr#'>

<cfexchangefilter name="subject" value="#msg_sub#">

</cfexchangemail>

<cfif usr_msgs.recordcount GTE 1>

info from cfexchangemail fields:<br>

<cfloop query="usr_msgs">

<cfoutput>

#usr_msgs.subject#<br> 

#usr_msgs.CC#<br> 

#usr_msgs.fromId#<br>

</cfoutput>

</cfloop>

info from cfexchangemail.internetHeaders fields:<br>

<cfloop query="usr_msgs">

<cfoutput>

#ReplaceList(usr_msgs.internetHeaders["from"][1], ">,<", ",", ",", ",")#<br>

#ReplaceList(usr_msgs.internetHeaders["to"][1], ">,<", ",", ",", ",")#<br>

#ReplaceList(usr_msgs.internetHeaders["cc"][1], ">,<", ",", ",", ",")#<br>

</cfoutput>

</cfloop>

</cfif>

 

You can reference the bugbase, for the enhancement request originally logged for this feature.


Many times you would want to tweak the performance of the ColdFusion server or want to debug the bottlenecks that make the server unresponsive.

To analyze this, ideally you would want to have Thead dumps and Server memory snapshot(Heap Space, Eden Space, Survivor Space, Old Gen, Perm Gen).

While you can use JDK tools like jstack to get the dumps, it is tedious to install it and schedule the thread dumps.

So, programmatically thread dumps and memory snapshots are are triggered and can be configured as a scheduler task and can be triggered on-demand as well.

Download the following zip file and move it to the server where you want to take thread dumps.

 

threaddump.zip

This zip file contains two files. One is threaddump.jar file.

Place this file under: C:\ColdFusion11\cfusion\wwwroot\WEB-INF\lib\ and restart the server for it to load.

And the other file is the cfm file takethreaddump.cfm where the call to ThreadDump class is made and the path where the dump content should be written.

By default it is dumped to the file #GetTempDirectory()#/threaddump<Day>-<Month>-<Year>.log

(Depending on the server location it translates similar to C:\ColdFusion11\cfusion\runtime\work\Catalina\localhost\tmp\threaddump12-8-2015.log)

You can change this to any other convenient path in the cfm file.

You can call this cfm on-demand at point of time or configure a new scheduled task to schedule it at some interval.

More number of Thread dumps are more helpful for problem analysis. So, it is better to take at some interval.

On every new day, dump is rotated automatically to a new file name.