Who’s On Phirst

All companies have different needs and reasons they pursue outsourcing. While Phurnace had success with our first outsourcing project, it was not without lessons learned. Having just kicked off our second outsourcing project, I'd like to share some of these lessons for those of you who might be considering an outsourced software development project.

Recognize the inevitable challenges
If you are going “offshore”, the language barrier is always a challenge. Even well spoken English can sometimes be hard to understand when you are accustomed to hearing the Texas drawl. The time difference is challenging primarily because it adds stress to the end of your day and in the same way your begins with stress. Finally, the lack of direct interaction is by far the biggest challenge. Nothing replaces direct communication. The recognition of these challenges is important so that you find creative ways to mitigate them.

Choose your project wisely
Selecting a project with a lot of unknowns leaves the door open for disaster. In addition, maintenance projects can be a bad idea. Maintenance is often perceived as easy which evolves to thinking that maintenance can be outsourced. While it is certainly possible, maintenance often involves very difficult problems and your most knowledgeable developers. Instead, find a project that your internal development staff could knock out of the park; but, is not the best use of their time, technically speaking.

For Phurnace, the best projects are usually those that have two qualities:

• Direct knowledge of the problem domain and scope by our development staff
• Well defined deliverables

Don’t underestimate the time required
View your outsource team as an addition to your current team. One common mistake is to fall for the sales pitch that they will manage it for you. If you just acquired an outsource team of five people, think about the impact of 2 or 3 new additions to your staff. Use that as a starting point for estimating your time required.

Daily Communication
Agile development practices encourages direct and often communication. This same approach works very well with an outsourced team. Setup daily calls to get your project going. It may require a little nudging, but be persistent. The outsource team will eventually give because they want the project to be successful, too. If you gain confidence, you can figure out what works best once you have an understanding of how your team interacts with the rest of your organization.

Do away with formal status reports
Formal status reports might as well be named TPS reports (from the movie Office Space). They are very overrated but extremely common in outsourcing processes. They usually involve a few tables with tasks and hours. This tells you nothing about the actual progress. Think about what you really want to know and tell them what you to know and when. Here are some examples: What issues are currently outstanding. What items is the outsourcing team waiting on you for. What are is the plan for next week by person. This is the information you need so problems can be resolved quickly, and you can hold each individual accountable as you would your own team.

Expect to be very technical
Know the code that they are working with, understand the technical challenges, and be capable of providing direct technical advice. Don’t delegate this on an as-needed basis. If this is not your expertise, my recommendation is to find someone who can be on this project from day 1 to provide this assistance. This is extremely important because otherwise you will never really know how well the project is going.

Don’t forget testing
Ask for tests as deliverables. More importantly, make sure these deliverables are something you can take in-house in the case you decide to no longer use the outsourcing team.

So those are a few of the lessons learned from my first time around at Phurnace. I am excited about my new team and look forward watching their project -- very closely!

In Agile Software Development
Comment (0) Read More...

Book review: Testing Extreme Programming by Lisa Crispin and Tip House

Testing Extreme Programming is part of the Addison-Wesley XP series of books. The most well known book in this series is probably Extreme Programming Explained: Embrace Change, by Kent Beck. Since one of the frustrating things about being a QA engineer or software tester on an agile or XP project is that there is hardly mention of the role of QA or the place of the tester in most XP literature, I decided to read this book.

If you do not know much about the tenets of XP, this is briefly explained in the beginning of the book, from a tester's point of view. There is no prerequisite to read Extreme Programming Explained. The first part of the book gives an overview of XP: communication, simplicity, feedback, and courage. Then it goes more in depth about the role of the tester in an XP project and why XP projects need testers. In examining unit testing versus acceptance testing, the authors propose that acceptance tests are not as easy for developers to write as it breaks from their natural workflow. Since this book was written in 2002, this obviously predates Fit for Developing Software: Framework for Integrated Tests by Rick Mugridge and Ward Cunningham. But I do think that, in my experience, developers do not have the same testing focus as a dedicated tester, so, in general, I agree with the authors.

The second part of the book takes the reader on a test drive of an XP project. The authors start the road trip with the release planning and iteration planning phases and explain how the tester can facilitate. Giving real-world examples, I believe their ideas and suggestions can be very helpful, especially to the tester who has never worked with XP. The authors then go into great detail describing how acceptance tests should be developed and automated. Using JUnit as an example, the authors show how to quickly automate acceptance tests. Here the authors make the assertion that Java is easy to write. I'm sure this is just to encourage the tester to dive in, but if the tester is technically weak, I'm not sure that the examples would be easily digested. A technically weak tester would need to pair with a developer to overcome this. But overall, the presentation is good.

The last section of the book explains how to use these ideas in less than ideal XP situations. Ideas are given for collaborating and communicating in large or distributed environments that are very helpful. Also, the authors conclude that you can use these ideas to gradually ease a project into XP, or even just for the test organization on a waterfall project.

One slight nag I have about this book is that the example project that is given in the book is a web application called XTrack (similar to XPlanner) at xptester.org. The xptester.org website is now defunct, but it would have been neat to actually view the website while reading the book. Also Brian Marick's old testing.com website is referenced (this is now exampler.com). But such is the danger of print. It would be great if there was a second edition, but it looks like Lisa Crispin has moved on to co-authoring another book.

In reading this book, I have made notes of concepts that I would like to explore in the future and hopefully apply to my day-to-day work. Overall, I'd recommend this book to both new and experienced testers. I rate this book 9 out of 10.

In Untagged 
Comment (0) Read More...

On September 26th, IBM will stop supporting WebSphere 5.1 and its various flavors. Nothing is particularly unusual about software companies ending support for aging products. But, the timing is terrible for customers who wish to upgrade to WebSphere 7.0.

IBM does provide an option for extending your support in a new “5 + 3” support policy. The policy has expanded the standard length of support to five years from three and the length of available extended support to three years from two.

Applying this new policy to WebSphere 6.1, we can expect IBM to reach end standard support on June 30, 2011. If one was to upgrade immediately to 6.1, you would have to migrate again in less than three years. Considering that most upgrades are measured in months, if not years, one could argue that three years is simply not enough time to amortize the upgrade costs. The end result: with the end of life of 5.1 happening before the general availability of 7.0, customers are feeling pressure to purchase the extended support to help them bridge to 7.0.

Moreover, the uncertainty of how long that migration will take is unnerving. Without a large investment of time and energy, one is unable to effectively estimate the migration effort. Simply put, most companies will have to leap before they look. Scary.

There is another option. Phurnace Deliver provides customers the ability to Snapshot a 5.1 instance and Deploy to a 6.1 instance. We will take your System Resources, Servers and other usual suspects, and migrate them to a 6.1 instance. Phurnace Deliver will even redeploy the Applications for you. You might find that you don’t need to rewrite any code and simply needed to move the EAR and System Resources to the new 6.1 instance. Or, you may be able to quickly identify problem areas to resolve before migration.

Either way, it’s a simple, cost effective way to have a look before you leap into an upgrade.

In WebSphere
Comment (0) Read More...

Anyone who’s worked with J2EE application servers has more than a passing familiarity with XML. It’s the weapon of choice for configuration in the 3 platforms I’ve spent most of my time working with; JBoss, WebLogic, and WebSphere. When I first started working at Phurnace, I would verify these files by hand using a diff tool that could handle single documents or a directory tree. I would compare a ‘known good’ configuration to the configuration produced by Phurnace Deliver. This worked great while I was getting my feet wet and finding my way around each platform’s configuration peculiarities.

For obvious reasons, this method sucked a lot after I got a handle on the different configuration conventions used by each platform. The process was incredibly monotonous and error prone. My mind would start to wander as I robotically clicked through the differences in dozens of XML files. Most of the time the differences were expected due to system generated unique IDs or environmental differences. I decided I needed to automate this process simply because I dreaded doing it manually. Because we were dealing with XML, XPath seemed to be the obvious choice for programmatic validation. So I just needed to whip up some XPath expressions and a simple Java class to resolve them and I was done. No bubbles, no troubles. Right?

Well I was forgetting that a WebSphere profile with one node and one server contains more than 100 individual XML files ranging in size from <1K to 600K+. If I thought using the diff tools was tedious, it was nothing compared to generating thousands of XPath expressions by hand. Plus, every time the product expanded into new areas of configuration, I was looking at hours of updating the automation data to keep it current. I needed a tool that would generate these XPath expressions for me. I had some specific needs that weren’t addressed by the sample classes and open source solutions I was finding on the web so I decided to write my own. Keep in mind:

 

• Coding isn’t my forte. You may find bone-headed logic or hacky work-arounds. But for my purposes it works and it’s quick. Consider it your chance to test the tester and file a bug by leaving a comment. I’d love to get feedback and improve it!
• This is a simplified version of the class I use and it works fine on simple XML. This sample class has some limitations. It doesn’t handle namespace designations and I haven’t added handling for special characters. I encourage you to play around with it and add methods that suit your purposes.

import java.io.File;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.parsers.ParserConfigurationException;
import javax.xml.xpath.XPath;
import javax.xml.xpath.XPathConstants;
import javax.xml.xpath.XPathExpressionException;
import javax.xml.xpath.XPathFactory;

import org.apache.commons.lang.StringUtils;
import org.w3c.dom.Document;
import org.w3c.dom.NamedNodeMap;
import org.w3c.dom.Node;
import org.w3c.dom.NodeList;
import org.xml.sax.SAXException;

public class XPathGenerator {

	/*
	 * fileList - a list of files to generate XPath expressions from xpath - an
	 * xpath instance used to verify generated xpaths
	 */
	private static List fileList = new ArrayList();
	private static XPath xpath = XPathFactory.newInstance().newXPath();

	/*
	 * Simple utility method that checks for whitespace at the beginning a text
	 * node
	 */
	private static boolean isWhiteSpace(String nodeText) {
		if (nodeText.startsWith("\r") || nodeText.startsWith("\t")
				|| nodeText.startsWith("\n") || nodeText.startsWith(" "))
			return true;
		else
			return false;
	}

	/*
	 * Simple utility method to verify brutishly assembled xpath expressions
	 */
	private static void checkXPath(String xpathExpression, Node node) {
		Object xpathCheck;
		try {
			xpathCheck = xpath.evaluate(xpathExpression, node
					.getOwnerDocument(), XPathConstants.BOOLEAN);
			if (xpathCheck.toString() == "true") {
				/*
				 * print the file/xpath combo to use for future verification For
				 * Example:
				 * file:/C:/tmp/sample.xml=/rootNode/firstChild/firstGrandChild
				 * [@attrib="value"][text()="some text"]
				 */
				System.out.println(node.getOwnerDocument().getDocumentURI()
						+ "=" + xpathExpression);
			}
		} catch (XPathExpressionException xpe) {
			System.out.println("\n\n" + xpathExpression);
			xpe.printStackTrace();
		}
	}

	/*
	 * Simple utility method to check for a text node on the currenrt XML
	 * element
	 */
	private static boolean hasValidText(Node node) {
		String textValue = node.getTextContent();

		return (textValue != null && textValue != ""
				&& isWhiteSpace(textValue) == false
				&& !StringUtils.isWhitespace(textValue) && node.hasChildNodes());
	}

	/*
	 * Simple utility to check for attributes on the current element
	 */
	private static boolean hasValidAttributes(Node node) {
		return (node.getAttributes().getLength() > 0);

	}

	/*
	 * Build XPath attribute list for individual XML Element Nodes Iterate
	 * through all attribute nodes and build a string that can be included in an
	 * XPath expression to validate an XML document. Resulting string will look
	 * like: [@attrib1="value1" and @attrib2="value2"] Skip this if the
	 * attribute list is empty.
	 */

	private static String buildAttribString(Node node, String pathExpr) {
		NamedNodeMap nnlist = node.getAttributes();
		pathExpr = pathExpr + "[";

		int attribCount = 0;
		// iterate over attributes
		for (int i = 0; i < nnlist.getLength(); i++) {
			// grab attribute name and value
			String attribName = nnlist.item(i).getNodeName();
			String attribValue = nnlist.item(i).getNodeValue();

			// if we've already added attributes to the path expression append
			// the current one
			if (attribCount > 0) {
				pathExpr = pathExpr + " and ";
			}

			pathExpr = pathExpr + "@" + attribName + "=\"" + attribValue + "\"";
			attribCount++;
		}

		pathExpr = pathExpr + "]";

		return pathExpr;
	}

	/*
	 * processNode checks for attributes and text nodes on an xml node and
	 * process them accordingly
	 */
	public static NodeList processNode(Node node) {

		if (hasValidAttributes(node) || hasValidText(node)) {
			String pathExpr = "/" + node.getNodeName();

			// check for attributes
			if (hasValidAttributes(node)) {
				pathExpr = buildAttribString(node, pathExpr);
			}

			// Make a copy of node to preserve it's state
			Node tmpNode = node;

			// Build pathExpr for XPath Expression by working backward through
			// the XML until we hit the document node.
			while (tmpNode.getParentNode() != null
				    && tmpNode.getParentNode().getNodeType() != Node.DOCUMENT_NODE) {

				tmpNode = tmpNode.getParentNode();
				String nodeName = tmpNode.getNodeName();

				if (hasValidAttributes(tmpNode)) {
					String attribString = buildAttribString(tmpNode, nodeName);
					pathExpr = "/" + attribString + pathExpr;
				} else {
					pathExpr = "/" + nodeName + pathExpr;
				}
			}

			if (hasValidText(node)) {

				/*
				 * Build XPath text value expression to verify text values for
				 * node. This is a little tricky because linebreaks, tabs and
				 * spaces included in the XML document are considered text
				 * nodes.
				 */

				// Copy original node to a textNode to preserve state of
				// original node for future operations
				Node textNode = node;

				/*
				 * This check iterates through child nodes of the original node
				 * until it reaches the next element node. As long as the node
				 * is not null, starts with white space and is not an element
				 * node, move on to the next node
				 */
				while (textNode != null
					      && isWhiteSpace(textNode.getTextContent()) == true
					      && StringUtils.isWhitespace(textNode.getTextContent())
					      && textNode.getNodeType() != Node.ELEMENT_NODE) {
					textNode = textNode.getFirstChild();
				}

				/*
				 * If the text content of the current node is not null, doesn't
				 * start with whitespace and has child nodes, we grab the text
				 * and frame it into an XPath text argument. A sample element
				 * and it's resulting text argument:
				 * 
				 * some text here YIELDS
				 * element1[text()="some text here"]
				 * 
				 * NOTE: The check for child nodes prevents us from creating a
				 * blank text argument for self contained elements (example:
				 * )
				 */
				pathExpr = pathExpr + "[text()=\""
			                   + StringUtils.strip(textNode.getTextContent()) + "\"]";

			}

			checkXPath(pathExpr, node);
		}
		// gather children and return
		return node.getChildNodes();
	}

	// This function takes a group of nodes and generates XPath expressions for
	// them until it runs out of nodes
	public static void processNodeList(NodeList nodelist) {
		for (int i = 0; i < nodelist.getLength(); i++) {
			if (nodelist.item(i).getNodeType() == Node.ELEMENT_NODE
			      && (hasValidAttributes(nodelist.item(i)) || hasValidText(nodelist
					   	.item(i)))) {
				processNode(nodelist.item(i));
			}
			processNodeList(nodelist.item(i).getChildNodes());
		}
	}

	// This function takes the cmd line argument and adds to fileList all files
	// in the directory and it's subdirectory
	private static void processDir(File dir) {
		File[] fileArray = dir.listFiles();

		for (int i = 0; i < fileArray.length; i++) {
			if (fileArray[i].isFile()) {
				if (fileArray[i].toString().endsWith(".xml")) {
					fileList.add(fileArray[i]);
				}
			} else if (fileArray[i].isDirectory()) {
				processDir(fileArray[i]);
			}
		}
	}

	// This starts the process of an individual XML file
	private static void processXML(File xmlFile) throws SAXException,
			IOException, ParserConfigurationException, XPathExpressionException {
		try {
			DocumentBuilderFactory factory = DocumentBuilderFactory
					.newInstance();
			factory.setIgnoringComments(true);

			DocumentBuilder parser = factory.newDocumentBuilder();
			Document doc = parser.parse(xmlFile);

			NodeList startlist = doc.getChildNodes();

			processNodeList(startlist);
		} catch (Exception e) {

		}
	}

	/*
	 * Entry point. Pass in the xml file or top level directory of a group of
	 * files to generate XPath expressions for them.
	 */

	public static void main(String[] args) {
		File startDir = new File(args[0]);
		try {
			if (startDir.isFile()) {
				processXML(startDir);
			} else if (startDir.isDirectory()) {
				processDir(startDir);
				for (int i = 0; i < fileList.size(); i++) {
					processXML(fileList.get(i));
				}
			}
		} catch (Exception e) {
			e.printStackTrace();
		}
	}
}

In Xpath Expressions, XML
Comment (1) Read More...

When I was younger, I considered running to stay in shape to be about as much fun as a root canal. Tennis, basketball, and softball occupied my time, as they provided the diversion of the particular ball that the sport involved. Now a days, however, I find myself enjoying the sheer simplicity of running. The uniform is simple, just shoes, shorts, and an optional shirt, and in Austin, you don't even have to leave your neighborhood to enjoy a great course. I'd like to share a couple of runs that I really enjoy.

Most of my running starts from the legendary "Rock", which is located underneath the Mopac foot bridge near Austin High. From here, one of my favorite runs is to head out Lake Austin Boulevard until it hits Scenic Drive. Scenic provides a very nice view of Lake Austin, and several nice, but very runnable hills. I like to follow Scenic until it hits Pecos Street, and then take a right. Pecos runs right through the heart of Tarrytown, which used to be considered a suburb in Old Austin. Pecos hits Enfield Road, which has a nice hill right before you get to Exposition Boulevard. From the top of this hill, you can see the Texas State Capitol in downtown Austin. Enfield crosses under Mopac again, where you can take a right and run through Clarksville, a neighborhood founded by freed slaves. Clarksville to me is the kind of neighborhood that really represents the funky-cool character of Austin. From the western edge of Clarksville, you are treated to a great view of the Texas Hill Country, right before you descend back to the Rock. This run comes in at just over 8 miles.

One of the newer runs that I just started doing in the last year is one that takes me south of the Colorado River. This is another great neighborhood in Austin, which despite fairly rapid gentrification, has still managed to maintain its look-and-feel. The absolute highlight of this run is a viewing of the Mary Street Goat. I kid you not, this is a live goat, complete with horns and a beard. Most mornings, he's out in the front yard of his owner's house, located near Congress Avenue. You can then take Congress north to the Lady Bird Lake Hike and Bike Trail, while enjoying a great view of the unofficial Main Street of Texas, and another view of the State Capitol. This run covers between 8-9 miles.

My early morning running puts me in the perfect frame of mind to write some kick-butt software here at Phurnace, while trying to avoid the middle-age spread. I've made some great friends, and I'm able to experience an Austin that a lot of people aren't fortunate enough to get to enjoy.

In Untagged 
Comment (0) Read More...