International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395 -0056

Volume: 05 Issue: 08 | August-2018 www.irjet.net p-ISSN: 23950072

AUTOMATED ERROR DETECTION FOR PYTHON USING JENKINS

1
Miss.V.Mohanakumari.,2 Ms. Kavitha.S .,M.C.A., M.Phil., (P.hd).

1M.phil Research Scholar, Department of computer Science, Auxilium College of Arts & Science for
Women (Autonomous), Gandhi Nagar, Vellore, Tamilnadu, India.

2Assistant Prof, Department of Computer Science , Auxilium College of Arts & Science for Women
(Autonomous) Gandhi Nagar, Vellore.

---------------------------------------------------------------------***---------------------------------------------------------------------
Abstract:
The Python programming language is
typically not seen as a language that can be
formally verified. This research attempts to bridge
the gap by introducing novel techniques to annotate
Python programs with type specifications,
contracts, and translate them to statically verifiable
components. In this proposed method a novel has
introduced a tool, Python Correct, which uses these
techniques to perform extended static checking
(ESC) on Python programs, as well as to generate
executable test cases through symbolic execution.
These analyses serve to improve code quality and
development productivity. One of the problems that
occur when writing contracts is that whenever the
program cannot be verified by an automatic verifier,
the cause could lie in three different reasons. Aim
to show that Python programs can benefit from
existing static verification tools and techniques if
they are simply made available to Python
developers. The goal of program verification is to
prove mathematically that the given program fulfils
a given formal specification. The consequence is
that for every input that fulfils the precondition,
post condition and all loop invariants have to hold.
The experimental result shows that accuracy of
error code and validates the run time process in the
framework. To reduce the minimum time of error
detection.
Introduction
© 2018, IRJET.NET- All Rights Reserved
Software testing is well established as
an essential part of the software development
process and as a quality assurance technique widely
used in industry. Furthermore, literature suggests
that 30 to 50% of a project’s effort is consumed by
testing. Developer testing (a developer test is “a
codified unit or integration test written by
developers”) in particular, has risen to be an
efficient method to detect defects early in the
development process. In the form of unit testing, its
popularity has been increasing as more
programming languages are supported by unit
testing frameworks (e.g., JUnit, NUnit, etc.).
The main goal of testing is the detection
of defects. Developer testing adds to this the ability
to point out where the defect occurs. The extent to
which detection of the cause of defects is possible
depends on the quality of the test suite. In addition,
Beck explains how developer testing can be used to
increase confidence in applying changes to the code
without causing parts of the system to break. This
extends the benefits of testing to include faster
implementation of new features or refactoring.
Consequently, it is reasonable to expect that there is
a relation between the quality of the test code of a
software system and the development team’s
performance in fixing defects and implementing
new features.
Python is a high-level, interpreted, interactive and
object-oriented scripting language. Python is
designed to be highly readable. It uses English
Page 1
 International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395 -0056
Volume: 05 Issue: 08 | August-2018
keywords frequently where as other languages use
punctuation, and it has fewer syntactical
constructions than other languages.
 Python is Interpreted − Python is
processed at runtime by the interpreter. You
do not need to compile your program before
executing it. This is similar to PERL and
PHP.
 Python is Interactive − You can actually sit
at a Python prompt and interact with the
interpreter directly to write your programs.
 Python is Object-Oriented − Python
supports Object-Oriented style or technique
of programming that encapsulates code
within objects.
 Python is a Beginner's Language − Python
is a great language for the beginner-level
programmers and supports the development
of a wide range of applications from simple
text processing to WWW browsers to
games.
Society is coming to use and rely more and more on
computer systems, such as safety Critical systems,
business systems, home appliance systems,
entertainment systems, and mobile systems. Our
dependence on such computer systems raises
people’s awareness of the social importance of the
software quality in these systems. The quality of
software is eventually determined when a product is
released. A product manager usually has to make
the decision on when to release a product using
collected software quality data; however, because
software is substantially complex, methods for
measuring software quality are sometimes in
adequate .Consequently, a product manager may be
pressured to make a release decision to meet the
product release schedule with uncertain software
quality. This thesis proposes a statistical software
testing technique to provide a rationale for
© 2018, IRJET.NET- All Rights Reserved
www.irjet.net p-ISSN: 23950072
software quality to help make this management
decision.
Software quality is measured with verification and
validation. Verification is a method used to ensure
that a product is built correctly. Validation is a
method to ensure that the correct product is built .
While both methods are important, this thesis will
focus on verification. Well-known software
verification techniques are testing, review, program
proving, and model checking. This thesis will focus
on testing. The number and importance of test cases
that are performed in testing and the absence of
errors during executing those test cases gives a
measure of the confidence of software quality.
The problem of testing is that the combinatorial
number of input values is exponential on
the sum of the number of input variable bits, which
is large; thus, testing all combinations of
input values is usually impractical. Hence, many
testing approaches to determine important
testing input value combinations have been
proposed. The black-box testing techniques
in particular deliver benefits for testing large and
complex software, since they only use
specifications that abstract away details of source
code.
The first issue of black-box testing is how to
identify use cases to test. A use case is a detailed
description of a sequence of transactions between
system and actors that exist outside the system.
Black-box testing lists testing input patterns that
can be derived from specifications, such as
requirement specifications or module interface
specifications. In industry, testers usually identify
use cases with the prediction of possible usages of
the product; however, since testers perform the
usage prediction individually and ad-hoc, the
prediction depends on the testers’ individual skills
and preferences, which can cause uncertain
Page 2
 International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395 -0056
Volume: 05 Issue: 08 | August-2018
software quality. Several researchers have
suggested methods for improving this use cases
identification.
Jacobson introduced a systematic process to
identify use cases. His process incorporated UML
use case diagrams and produced the use cases.
However, since the description of use cases is
written in a natural language, which has ambiguity,
the exhaustive listing of all use cases is difficult.
Some researchers proposed identification methods
using formal specifications such as Z[2], VDM
[1],and state machine . By utilizing its formalism,
they could generate an exhaustive list of use cases
that are specified. However, since most
specification languages are difficult to write
because of their complicated model construction,
they are rarely adopted in industry.
Among them, Trace Function Method (TFM) is
reportedly beneficial . It is formal but relatively
intuitive, conforms to the information hiding
principle and has reader-friendly tabular format. In
this thesis, we identify use cases with TFM
specifications.
The second issue of black-box testing is how to
prioritize test cases. The exhaustive listing of use
cases of a large software component yields an
intractable number of use cases. Development
organizations have a limited time for testing cases;
thus, we have to prioritize them to reduce the
number of test cases to perform. Whittaker
modeled the system as a discrete-time Markov
chain where the transitions are user operations, and
generated prioritized test cases with a random
choice of transitions in the Markov chain; however,
they left the discussion on how to identify states
and transitions of the Markov chain as an open
question.
© 2018, IRJET.NET- All Rights Reserved
www.irjet.net p-ISSN: 23950072
It can be generated prioritized test cases from a list
of pairs (event, class of system’s executed
events)with an occurrence probability. The event is
an invocation of an access program of a component.
Although she introduced classes of executed events,
she left room to discuss how to identify those
classes. Musa proposed the operational profile,
which profiles each user operation with its
occurrence probability and importance and
prioritizes user operations to test. Although this
approach is reportedly beneficial the identification
of the set of user operations relies on an informal
refinement process, which makes completing an
exhaustive list of user operations a complex task.
Testing process:
Python Features
Python's features include −
 Easy-to-learn − Python has few keywords,
simple structure, and a clearly defined
syntax. This allows the student to pick up
the language quickly.
 Easy-to-read − Python code is more clearly
defined and visible to the eyes.
 Easy-to-maintain − Python's source code is
fairly easy-to-maintain.
Page 3
 International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395 -0056
Volume: 05 Issue: 08 | August-2018
 A broad standard library − Python's bulk
of the library is very portable and cross-
platform compatible on UNIX, Windows,
and Macintosh.
 Interactive Mode − Python has support for
an interactive mode which allows
interactive testing and debugging of
snippets of code.
 Portable − Python can run on a wide
variety of hardware platforms and has the
same interface on all platforms.
 Extendable − You can add low-level
modules to the Python interpreter. These
modules enable programmers to add to or
customize their tools to be more efficient.
 Databases − Python provides interfaces to
all major commercial databases.
 GUI Programming − Python supports GUI
applications that can be created and ported
to many system calls, libraries and windows
systems, such as Windows MFC,
Macintosh, and the X Window system of
Unix.
 Scalable − Python provides a better
structure and support for large programs
than shell scripting.
Literature Survey:
Test case generation is often cited as one of
the most challenging tasks in testing dependable
systems. Besides benefits as a verification
technique by its own right, model checking is
emerging as an efficient method for automating test
case generation. Existing testing criteria and a range
of new criteria, namely the vacuity-based ones,
inspired by formal requirements have been used in
© 2018, IRJET.NET- All Rights Reserved
www.irjet.net p-ISSN: 23950072
model checking- assisted test generation. This paper
reviews some of these existing and new test criteria.
We developed a unified framework for evaluating
the effectiveness of these test criteria and the
efficiency of model-checking-assisted test
generation for these criteria. The benefits of this
work are three-fold: first, the computational study
carried out in this work assesses the practical
effectiveness and efficiency of model-checking-
assisted test case generation, which are important
metrics to consider for selecting the right test
criteria and test generation approach. Second, we
propose a unified test generation framework based
on generalized B¨uchi automata. The framework
uses the same off-the-shelf model checker, in this
case, SPIN model checker [10], to generate test
cases for different criteria and compare themon a
consistent basis. Last but not least, we describe in
great details the methodology and automated test
generation environment that we developed on the
basis of our unified framework. Such details are of
interest to researchers who needs to carry out their
own experimental study on test criteria, and to
practitioners who want to integrate model checking
assisted test generation into their testing process.
Python provides two very important features
to handle any unexpected error in your Python
programs and to add debugging capabilities in
them:
– Exception Handling: This would be
covered in this tutorial.
– Assertions: This would be covered
in Assertions in Python.
• An exception is an event, which occurs
during the execution of a program, that
disrupts the normal flow of the program's
instructions.
• In general, when a Python script encounters
a situation that it can't cope with, it raises an
exception. An exception is a Python object
that represents an error.
Page 4
 International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395 -0056
Volume: 05 Issue: 08 | August-2018 www.irjet.net p-ISSN: 23950072

• When a Python script raises an exception, it high quality is able to cover each branch of the data
must either handle the exception flow graph of the program. Each input parameter
immediately otherwise it would terminate has to be tested and a corresponding testing method
and come out. should be available in the unit test.
• There are two main error handling models:
status codes and exceptions. Status codes The generation of the error is done normally
can be used by any programming language. automatically by specific mutation testing software
Exceptions require language/runtime tools. This thesis describes an approach for
support. mutation testing in the programming language
Python. The tool which is used for mutant
– Raising Exceptions generation is the proof-of-concept tool MutPy. In
the event that a mutated version of the program has
– Catching Exceptions been discovered by the existing testing data, the
mutant is no longer needed for further testing and
– Swallow the Exception the tester can discard it. We can remove the mutant
from the test results, because the output, which it
– Dealing With Transient Failure
delivers, is not the correctly stated one we expect.
We refer to this operation as killing the mutant. The
– Error Logger
second goal of mutation testing is to have as many
errors as possible killed. Unfortunately, in many
cases the software, used for mutant generation,
tends to create errors that are semantically different
from the original program, but are, nevertheless,
functionally equivalent to it. They are called
Functionally Equivalent Error or simply Equivalent
Error.

Python

Python provides a built-in assert function

The tester is required to generate such test
whose arguments closely resemble the
data, so that these introduced changes, which can be
aforementioned “assertion, along with some form
referred to as errors as well, can be detected. This
of identifier”:
we achieve by comparing the outputs of the original
program and the mutated version. If the output Assert test_condition, "Test description"
differs, we have discovered the errors. Each
different version is called a mutant. In the process When an assertion is false, an exception of type
of mutation testing, the tester executes many errors Assertion Error is raised associated with the
against the existing unit or module tests. The main string Test description. Uncaught, this will
goal of the mutation testing s to evaluate the quality cause our Python test to exit with a non-zero status
of the existing tests. Under the term tests we and a stack-trace printed to STDERR, like any
understand test program and test data. A test data of other exception.

© 2018, IRJET.NET- All Rights Reserved Page 5

 International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395 -0056
Volume: 05 Issue: 08 | August-2018
Assertion Error has taken on a special signifying
value for Python testing tools – virtually all of them
will provide testing assertions which raise
Assertion Errors, and will attempt to catch
Assertion Errors, treating them as failures rather
than exceptions. The unit test* documentation
provides quite a nice description of this distinction:
Like Test::More‘s ok(), there’s no default useful
debugging information provided by assert other
than what you provide. One can explicitly add it to
the assert statement’s name, which is evaluated at
run-time:
assert expected == actual, "%s != %s" %
(repr(expected), repr(actual))
However, evidence that assert was not
really meant for software testing starts to emerge as
one digs deeper. It’s stripped out when one is
running the code in production mode, and there’s
no built-in mechanism for seeing how many times
assert was called – code runs that are devoid of
any assert statement are indistinguishable from
code in which every assert statement passed. This
makes it an unsuitable building block for testing
tools, despite its initial promise.
PyTest – a “mature full-featured Python testing
tool” – deals with this by essentially turning assert
in to a macro. Testing libraries imported by code
running under PyTest will have their assert
statements rewritten on the fly to produce testing
code capable of useful diagnostics and
instrumentation. Running:
Assert expected == actual, "Test description"
Using python directly gives us a simple error:
Trace back (most recent call last):
File "assert_test.py", line 4, in <module>
Assert expected == actual, "Test description"
© 2018, IRJET.NET- All Rights Reserved
www.irjet.net p-ISSN: 23950072
Assertion Error: Test description
Where running it under py.test gives us proper
diagnostic output:
assert_test.py:4: in <module>
assert expected == actual, "Test description"
E AssertionError: Test description
E assert 'Foo' == 'Bar'
unittest, an SUnit descendent that’s bundled as part
of Python’s core library, provides its own functions
that directly raise AssertionErrors. The basic unit is
assertTrue:
self.assertTrue(testCondition )
unittest‘s test assertions are meant to be run inside
xUnit-style Test Cases which are run inside
try/except blocks which will catch
AssertionErrors. Amongst others, unittest provides
an assertEqual:
self.assertEqual( expected, actual )
Although surprisingly we’ve had to remove the test
name, so that it’ll be automatically set to a useful
diagnostic message:
Traceback (most recent call last):
File "test_fizzbuzz.py", line 11, in test_basics
self.assertEqual( expected, actual )
AssertionError: 'Foo' != 'Bar'
Much as Perl unifies around TAP and Test::Builder,
Python’s test tools unify around the raising of
AssertionErrors. One notable practical difference
between these approaches is that a single test
assertion failure in Python will cause other test
assertions in the same try/except scope not to be
run – and in fact, not even acknowledged. Running:
assert 0, "Should be 1"
Page 6
 International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395 -0056
Volume: 05 Issue: 08 | August-2018
assert 1, "Should also be 1"
Gives us no information at all about our second
assertion.
RESULT ANALYSIS:
1) Developer Stage errors
If a program is syntactically correct – that
is, free of syntax errors – it will be run by the
Python interpreter. However, the program may exit
unexpectedly during execution if it encounters a
Developer stage error – a problem which was not
detected when the program was parsed, but is only
revealed when a particular line is executed. When a
program comes to a halt because of a runtime error,
we say that it has crashed.
Consider the English instruction flap your arms and
fly to India. While the instruction is structurally
correct and you can understand its meaning
perfectly, it is impossible for you to follow it.
Some examples of Python Developer errors
 Division by zero
 Performing an operation on incompatible
types
 Using an identifier which has not been
defined
 Accessing a list element, dictionary value or
object attribute which doesn’t exist
 Trying to access a file which doesn’t exist
Runtime errors often creep in if you don’t consider
all possible values that a variable could contain,
especially when you are processing user input. You
© 2018, IRJET.NET- All Rights Reserved
www.irjet.net p-ISSN: 23950072
should always try to add checks to your code to
make sure that it can deal with bad input and edge
cases gracefully. We will look at this in more detail
in the chapter about exception handling.
2) Logical errors
Logical errors are the most difficult to fix.
They occur when the program runs without
crashing, but produces an incorrect result. The error
is caused by a mistake in the program’s logic. You
won’t get an error message, because no syntax or
runtime error has occurred. You will have to find
the problem on your own by reviewing all the
relevant parts of your code – although some tools
can flag suspicious code which looks like it could
cause unexpected behaviour.
Sometimes there can be absolutely nothing wrong
with your Python implementation of an algorithm –
the algorithm itself can be incorrect. However,
more frequently these kinds of errors are caused by
programmer carelessness. Here are some examples
of mistakes which lead to logical errors:
 Using the wrong variable name
 Indenting a block to the wrong level
 Using integer division instead of
floating-point division
 Getting operator precedence wrong
 Making a mistake in a boolean
expression
 Off-by-one, and other numerical errors
Validation Results
Page 7
 International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395 -0056
Volume: 05 Issue: 08 | August-2018 www.irjet.net p-ISSN: 23950072

CONCLUSION

Programming languages have different

styles and paradigms with different advantages and
disadvantages. Many efforts have been put into
mapping C++ and java programming to Object, but
unfortunately the popularity and object-oriented
programming paradigm takes most attention on this
Fig 5.5: Measuring the Python Developing stage mapping. Python is a multi paradigms
Analysis programming language, dynamic typing, high level
built-in data types, and many add-on packages, it
incorporates predicate calculus, mathematical
proving, set theory and a lot of libraries. In addition
to that, it can be extended to contain new notations
and features. Model-based testing can encourage
different approaches to testing. It encourages on-
the-fly testing --- but in general, on the-fly test runs
are not reproducible, due to non determinism. It
suggests extending testing to noninvasive
monitoring or runtime verification --- if the test
harness supports observable actions, the test runner
can check log files or monitor network traffic for
conformance violations. The most prospect might
be better integration of design and analysis with
Fig 5.6 : Average precision on the Code testing. Exploration with composition is like model
Collection checking; it can check for safety, live ness, and
temporal properties. And, the same models are used
for these analyses as for automated testing.
Moreover, the models are written in the same
language as the implementation, which could make
them accessible to developers and test engineers,
not just formal methods experts.

FUTURE SCOPE:

Python programming language is

undoubtedly dominating the other languages when
future technologies like Artificial Intelligence(AI)
comes into the play. The future scope of python
Fig 5.7 : Code Regeneration & Recall validate
programming language can also be predicted by the

© 2018, IRJET.NET- All Rights Reserved Page 8

 International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395 -0056
Volume: 05 Issue: 08 | August-2018
way it has helped big data technology to grow.
Python has been an important part of Google since
the beginning, and remains so as the system grows
and evolves. Today, dozens of Google engineers
use Python, and we're looking for more people with
skills in this language.
REFERENCES:
[1] M. Najafi, H. Haghighi, (2012), “An approach
to animate Object-Z specifications using C+
+”,Scientia Iranica, Volume 19, Issue 6, Pages
1699–1721.
[2] Mehrnaz Najafia, Hassan Haghighia, (WCIT-
2011), “An approach to develop C++ Code from
Object-Z specifications”, 2nd World Conference on
Information Technology.
[3] Fukagawa, M., Hikita, T. and Yamazaki, H.,
(1994), “A mapping system from Object-Z to C+
+”,First AsiaPacic Software Engineering
Conference (APSEC94), IEEE Computer Society
Press, pp.220-228,1994.
[4] M. Najafi and H. Haghighi, ( 2011), “An
Animation Approach to Develop C++Code from
Object-ZSpecifications”, In International
Symposium on Computer Science and Software
Engineering, pp. 9-16.
[5] Sowmiya Ramkarthik and Cui Zhang, (2006 ),
“Generating Java Skeletal Code with Design
Contracts from Specifications in a Subset of Object
Z”, Proceedings of the IEEE/ACIS International
© 2018, IRJET.NET- All Rights Reserved
www.irjet.net p-ISSN: 23950072
Conference on Computer and Information Science,
Honolulu, Hawaii, pp. 405-411.
[6] Shengchao Qin, Guanhua He, (200 ), “Linking
Object-Z with Spec#”, 12th IEEE
InternationalConference on Engineering Complex
Computer Systems.
[7] Winfried Dulz. MaTeLo-statistical usage testing
by annotated sequence diagrams, Markovchains and
TTCN-3. Quality Software, 2003. Proceedings.
Third, page 336, November 2003.
[8] IK ElFar. “Model-based Software Testing.
Encyclopedia of Software”, pages 1–22, 2001.
[9] Constance L Heitmeyer and John D Mclean.”
Abstract Requirements Specification A New
Approach and Its Application”. (5):580–590,
1983.64
[10] William Hetzel. “The Complete Guide to
Software Testing.” Wiley; 2 edition, 1993.
[11] Daniel Hoffman.” Module test case
generation. AcmSigsoft Software Engineering”
Notes, pages 97–102, 1989.
[12] W E Howden. “A functional approach to
program testing and analysis. IEEE Transactions
on Software Engineering”, 12(10):997–1005,
October 1986.
Page 9