Working with STAF (part 2)

Well we have discussed some of the concepts of staf in post WAORKING WITH STAF. NOW we will have some thing interesting interaction by using JMeter.

Now let us know the concepts of Jmeter inorder to go further.

JMeter is a open source tool we can download from the site Apache Jmeter.
Jmeter is a performance tool used for finding the avg. response time, devaition etc parameters like the other tools load runner. The only use of the Jmeter is a open source and also it is a less memory eater when compared to the other tools.

NOW create a test plan according to the manual given in the Apache Jakarata site site itself, After creating a test plan save the file with desired name. LETs go to STAF now.

IF for suppose this file is in the remote system. we can execute this file and and the results can be obtained in our systems. INORDER to do this we have to give the trust levels to the remote systems. There are 5 trust levels, according to each trust level the access is differnet for the remote systems. After giving the trust level. Now we can execute the test plan by making a batch file.

In this Batch file we need the controller where the system we handle load generators where the test case is present, Well, i will make batch file very simple to understand and an idea how to execute the test plan by using the batch file.


REM set the controllers
REM set the load generators
REM set the plan

SET controller= webserver1
SET load = webserver2
SET plan=myplan.jmx

REM checking the STAF connecting to the remote systems

FOR %%H IN (%controller% %load%) DO STAF %%H PING PING

REM executing the test plan using jmeter

for %%H IN (%load%) Do staf %%H PROCESS START SHELL COMMAND C:\jmeter\bin\jmeter-n.cmd C:\jmeter\bin\myplan.jmx

REM copying the results from the remote system to our system

for %%H IN (%load%) Do staf %%H fs copy directory c:\jmeter\bin todirectory c:\logs\ tomachine %controller% EXT log CASEINSENSITIVE


This is a small batch file where we can use staf and make wonders.


Dear Upcoming Testers...

Testers can't go away with "click here,click there and see the result" kind of a thing while testing. Testers always need the "DEEP THINKING" while they test. One of the famous example given by Mr.James Bach which i already mentioned in About test cases. Testers always need Deep thinking.I have spoken with some of the Upcoming testers, they are in situation what is the role of a tester after getting into the company ! i don't want to pinpoint saying that "here goes the wrong so you don't know the role of a tester". I meet some of my friends who are trained like me, we are trained in different schools in different arts of styles but final output we get is knowledge. We also have a discussion several times, what is the job/role of testers in a company ? we always have to compromise by saying one thing or the other in our battle.
And more important most of the upcoming tester think automation means "QTP.LOAD Runner". This is not at all the correct way of thinking, automation includes many concepts to discuss tools and as far as approach. for example spell check in MS Office or in some application can also called as automated way. we have resources and the only thing is we have to improve our knowledge by implementing and sharing it. i got few examples for testers,

f'ing counting

interactive puzzle

The above examples are really awesome and we can also correlate this examples with Testing.

Working with STAF

STAF (software testing automation framework) is designed for reusable components called services. STAF is a remote agent used to control the test on various machines.for making easier to create automated test cases and workloads. it works on Windows,Linux,AS/400,MVS.

Concepts of STAF

1).STAFProc

2).STAF services
a)Internal services are within the STAFProc which means they are always available
and having the constant names.
b)External services are outside the STAFProc which means they are always from
outside for example from the java,c++ etc.
c)Custom services are always the external services then can also be written in our
custom and can be plugged into STAF.

3).Queues and handles

4).Variables

5).Security

6).Submitting Staf requests.

Configuration file:
After installing the Staf, in order to get the access to local machines, trust level has to be given and java sdk is needed for running the staf.


TRUST LEVEL 5 MACHINE tcp://local machine name or ip address.*


you can find the STAF.CFG file in the bin folder of STAF. You can also alter many features in the configuration file like

1).Specify the network interfaces

2).Define operational parameters

3).Define global variables

4).Specify the security access

5).Define startup/shutdown process

6).Enable and configure tracing

7).Register and configure external services

The idea behind STAF is to run a very simple agent on all the machines that participate in the STAF testbed. Every machine can then run services on any other machine, subject to a so-called trust level. In practice, one machine will act as what I called the 'test management' machine, and will coordinate the test runs by sending jobs to the test clients. STAX is one of the services offered on top of the low-level STAF plumbing. It greatly facilitates the distribution of jobs to the test clients and the collection and logging of test results. STAX jobs are XML files spiced up with special tags that contain Python code (actually Jython, but there are no differences for the purpose of this tutorial). This in itself was for us a major reason for choosing STAF over other solutions.

Common mistakes of software developers (cont.)

Some of the common problems are listed:

MY PROBLEM IS DIFFERENT

Many designers and programmers refuse to listen to the experiences of others, claiming that their application is different, and of course much more complicated. Designers should be more open-minded about the similarities in their work. In response, ask “what is different in the LCD display software in a cellular phone versus one on a temperature controller? Are they really different?” Comparing control and communication systems side-by-side, both are characterized by modules that have inputs and outputs, with a function that maps the input to the output. A 256 by 256 image processed by a algorithm might not be very different from graphical code for a LCD dot matrix display of size 320 by 200. Furthermore, both use hardware with limited memory and processing power relative to the size of the application; both require development of software on a platform other than the target, and many of the issues in developing software for a micro-controller. The timing and volume of data is different. But if the system is designed correctly, these are just variables in equations. Methods to analyze resources such as memory and processing time are the same, both may require similar real-time scheduling, and both may also have high-speed interrupt handlers that can cause priority inversion. Perhaps if control systems and communication systems are similar, so are two different control applications or two different communication systems. Every application is unique, but more often than not the procedure to specify, designs, and build the software is the same. Embedded software designers should learn as much as possible from the experiences of others, and not shrug off experience just because it was acquired in a different application area.


Large if-then-else and case statements

It is not uncommon to see large if-else statements or case statements in embedded code. These are problematic from three perspectives

1.) They are extremely difficult to test, because code ends up having so many different paths. If statements are nested it becomes even more complicated.

2.). The difference between best-case and worst-case execution time becomes
significant. This leads either to under-utilizing the CPU, or possibilities of timing errors when the longest path is taken.

3.) The difficulty of structure code coverage testing grows exponentially with the number of branches, thus branches should be minimized.

This example confuses new testers who lack in programming experience.
Developers think their code is always correct and as mentioned earlier 99 % errors are corrected by themselves and remaining 1%.errors will be found out by testers. In the below example the

IF (0 < x < 12) then
SYSTEM.OUT.PRINTLN (“Month is” & i);
Else
SYSTEM.OUT.PRINTLN (“Invalid input”);


Consider how this code could fail. Here are some of the simple programming errors that are very common mistakes that can go wrong:

a) Suppose the programmers said less than or equals instead of less than. The program would eject 0 as bad character. The only way to catch the error I by testing with 0.

b) If the code is written as less than 12 instead of less than or equal to 12, the program would go wrong.


“Testers with just the four boundary characters, /, 0, 9, and: will reveal every classification
Error that the programmer could make by getting an inequality wrong or by mistyping”

Error Handling:

Errors in dealing with errors are common. Error handling errors include failure to anticipate the possibility of errors and protect against them, failure to notice error conditions, and failure to deal with a detected error in a reasonable way. Many programmers correctly detect errors but then branch into untested error recovery routines. These routines’ bugs can cause more damage than the original problem.

Some times the errors are more even large while executing the tests and the Microsoft’s worst scenario is we can’t copy the error messages. There are some tools for copying the text of such error messages and also we can take the screen shots.


Conclusion:


Testers should have a common sense in smelling the non obvious things. There is a relation between developers and testers in fixing the bugs. The probability of fixing the bugs is always depends on the way the test cases are communicated. Test cases play a major role in the QA's life. Test Cases are written in such a way that they are traceable, self contained and should not be duplicated in preparing they should always be atomic.



"Why go into something to test the waters? Go into it to make waves"





HAPPY TESTING

Common mistakes of Software developers

Introduction:

Most Software developers are not even aware that there favorites methods are problematic. Quite often experts are self thought, hence they tend to have the same bad habit as when they first began, usually because they never witnessed the better ways of performing their embedded systems. These experts then train novices who subsequently acquire the same bad habits. The purpose of this presentation is to improve the awareness to common problems, and to provide a start towards eliminating mistakes and thus creating software that is more reliable and easier to maintain.

It is easy for spending a million on testing a program. Common estimation of the cost of finding and fixing the errors in program range from 40% to 80% of total development cost. Companies don’t spend this kind of money to “verify that a program works”. They spend it because the program doesn’t work, it has bugs and they want them found. No matter what development methodology they follow, their programs still end up with bugs. Beizer’s (1990) review estimation the average number of errors in program released to testing at 1 to 3 bugs per 100 executable statements. There are big differences between programmers, but no one’s work is error-free.

One error per 100 statements is an estimate of public bugs; the ones still left in program after the programmer declares it error-free Beizer (1984) reported his private bug rate, how many mistakes he made in designing and coding a program, as 1.5 errors per executable statement. This includes all mistakes including typing errors.

“At this rate, if your programming language allows one executable statements per lines, you make 150 errors while writing a 100 lines program.”

Most programmers catch and fix more than 99% of their mistake before releasing a program for testing. Having found so many, no wonder they think they most found a lot. But they haven’t .Tester’s job is to find the remaining 1%.

Correcting just one of the mistakes within a project can lead to week or months of savings in manpower (especially during the maintenance place of a software life cycle.).