Novell’s software products have always undergone rigorous inhouse QA tests before going to market. But dynaTrace‚Äôs tools are turning the testing process into a relative piece of cake, according to Mike Demastrie, engineering manager at Novell.
Back in 2007, Novell started using the Development and Test Center editions of PurePath in attempts to raise efficiency around identifying, diagnosing, and resolving application performance issues in Novell’s identity management software.
Novell also explored several other toolsets at around that time, including products from OpNet and Wiley Software Testing, for example. Ultimately, however, Novell settled on dynaTrace, since it was the only candidate with all three of the capabilities needed: application performance monitoring, debugging, and profiling.
By using dynaTrace as part of its agile development efforts, the Linux distributor has doubled and even tripled performance test throughput without adding more staff, according to the engineering manager.
With the decline of the economy since late last year, these productivity gains have taken on even more importance.
Earlier on, when members of Novell’s QA team discovered performance issues, they had to bring the developer directly into the testing process right away. Back then, Novell’s QA testers and developers used a synchronous approach, in which two people worked together simultaneously on every issue.
The tester induced the problem, and the developer then fixed it. Both people used the same machine during these procedures, with one person sitting idly by and waiting. But this tied up hardware, as well as the time of testers and developers alike.
In contrast, dynaTrace’s tools can preserve individual transactions for later view, allowing developers to resolve software problems whenever it’s convenient for them, explained Lucy Monahan, QA engineer at Novell.
Monahan finds this asynchronous approach to be particularly helpful in working with identity management specialists, since these expert developers can get very busy, to say the least.
The tools use dynaTrace’s PurePath transaction tracing technology to help assess the “what, where, and why” of a problem, either from an end-user perspective (such as poor response time) or a code perspective (such as a database exception).
To find the root cause of a problem, PurePath uses techniques that include problem documentation; incident triage and problem isolation; deep diagnostics in realtime; and online problem reconstruction. The PurePath technology helps to analyze memory; code-level performance; database access; SOA/remote overhead; thread and synchronization; and exception/log discovery, for instance.
In one test with PurePath, for example, Novell discovered that a memory leak had caused the Java Virtual Machine (JVM) to hit an OutOfMemory condition. The QA engineer captured the needed information and sent it to the developers, who then quickly repaired the bug.
Available in two other editions, as well, dynaTrace’s tools support both J2EE and .NET development environments. The toolsets offer a number of additional capabilities, too, including runtime modeling, scaleability tuning, and 24-by-7 transaction monitoring and management.
Demastrie and Monahan also pointed to further advantages of dynaTrace. For one thing, Novell no longer needs to keep re-running certain tests in order to capture enough log and tracing information. although it’s still quite possible to do so.
Monahan said she typically performs more fine-grained analysis after first running a load test to pinpoint the areas of greatest interest. She likes the way that dynaTrace‚Äôs drilldown tools can be turned “on” for granular analysis, or “off” if they‚Äôre adding too much overhead to the process.
Through dynaTrace’s profiling feature, users can set up criteria for specific tests, making it easier to keep re-running the same test over time.
The dynaTrace tools also include charting and diffing features for verifying to developers that code fixes have solved the problem. Monahan cited pie charts that show API execution times, remote host connections, and the use of databases, threads, and memory dumps, for instance.
Charts on API execution provide visual depictions of how much time is being spent in hibernation, or on remote calling, for example. Lots of remote calling might indicate that an overabundance of “chattiness” is bogging down application performance.
Through the diffing feature, testers and developers can see how the results of specified tests have changed during successive software builds.
Monahan likes dynaTrace’s ability to display information in either PDF or HTML, for co-workers who don‚Äôt have DynaTrace installed on their systems. “It would be even better, though, if you also had the flexibility to export information as text files,” she suggested.
Also on Monahan’s wish list for future improvements in dynaTrace are some general usability enhancements, along with more messaging capabilities. But on the whole, she’s found the tools to be a huge timesaver, both for herself and her colleagues at Novell.
“This technology is very empowering. These days, I can give the developers test results that bring them to the root cause of a problem right away,” noted the QA engineer.
