Macro-based QA (June 2010)

Creating New Macros

To record a QA macro, type $record MyMacro.mac in the ZW3D text input area at
the bottom of the ZW3D window. You will then be prompted for a directory to put
the macro in. On a standard ZW3D development machine, the default directory will
be Source\Macro. Selecting a sub-directory within this directory will not
actually create the macro on the network drive, instead, it will create the same
directory structure in the current ZW3D run directory and place the macro there.
If you are not running on a standard ZW3D development machine, you need to
create a "Macro" folder in your ZW3D user folder (C:\Documents and Settings\ZW3D
Corp\User-14.50) and select it when prompted for a destination directory after
executing $record.
You can add sub-folders to the "Macro" folder to organize
your QA macros.
After selecting a directory, ZW3D will start a new ZW3D session and initialize a
file named "MyMacro.mac". Subsequent commands (most of them) executed in ZW3D
will be recorded to the file. After starting the recording, the standard
procedure for recording a macro that tests a PCR would be to:
1) Open the file associated with the PCR
2) Execute the commands that were failing and are now fixed
3) Insert commands in the macro to record various measures of the model.
4) Terminate the recording by typing $record again.
5) Type $qa (detailed below) to execute the new macro and record the baseline
output (.bl file).
6) Execute the macro again to make sure that it passes with the newly
generated baseline file.
The Qa Output menu (displayed on the right side of the screen when macro
recording starts) gives options for recording certain data about the active
model to the baseline file. Most of the commands documented below are available
from the form. Optional strings are prompted for during macro
recording. If you close the Output form or need it for some
reason, type $qaOut from the ZW3D command line.
The macro commands that begin with [QaOut..] write various types
of information to an output file.
The first time this output
file is generated, it becomes the baseline file for the macro.
The baseline file (.bl) is deposited in the macro's parent
directory.
Subsequent execution of the macro puts the output
file in "./QaOutputFiles".
Each time the macro is executed,
the new output file is automatically compared to the baseline
file.
If the files are different, the macro is tagged as
"FAILED" and a message is logged indicating which line of output
data is different from the baseline data.
The "offending"
output data is left in "./QaOutputFiles" to help you figure out
what went wrong.
Often, you will also get the macro file line
number where things went bad.
The buttons on the QaOutput form record the following commands
in the macro. See the command documentation below for details
about what will be tested.

[QaOutBary,include_blanked,<point|symbol|assembly>]
(prompts for
include_blanked and entity type)
[QaOutEntCnt,ignore_blank,<layer_name>]
[QaOutHist]
[QaOutHoles]
[QaOutLayers]
[QaOutVol,<percentage>]
[QaOutPnt,label,x,y,z]
(prompts for label and a pick for the point)
[QaOutDist,label,double_precision_distance]
(prompts for label and 2
entities to
calculate distance between )
[QaOutDbl2,label,double_precision_value, <percentage>] (prompts for label
and curve)
[QaOutSeparator]
[QaOutCAM]
[QaOutCAM,<multi>]
[QaOutLinks]
[QaOutRoot]
[QaOutDispAttr](prompts for entity)
[QaOutVar]
[QaOutComp]
[QaOutDim]
The description of each [QaOut...] macro command describes the type of data it
logs and how that data is compared to the baseline. It is up to the QA test
developer to decide what form of output is appropriate to test a given
functionality. The most commonly used tests are Volume, Barycentric, Entity
Count, and CAM Output.
A QA macro doesn't have to log output/baseline data, but they usually do.
macro can compare against theoretical results calculated within the macro
without ever referencing an external support file.
If the macro doesn't
generate the desired results, call [ExitMacro].

The

Almost everything can be tested by simply using the macro recorder and verifying
the results using the [QaOut] commands. The ZW3D macro language itself is a
much richer language supporting loops and conditional execution. See the
Vxmacro.DOC document for a full description.

Running Existing Macros

To execute existing macros, run "$qa" from the ZW3D command line.
shown below will be displayed.

The form

1
B

D
4

H
I

J
K
A) This Combo Box controls what kind of test will be executed when one of the
buttons is pushed
1) Execute Run macros reporting success or failure
2) Update Run macros updating the baseline result files
3) Leakage Run macros and check for object leakage in the database.
Failures are reported in QaLofFile.
4) Analyze Run macros many times looking for slowdowns and generate
extremely rough performance data in the file QaPerformance
5) Test Undo Run macros and check that all operations successfully
stored the Undo/Redo data. Failures are reported in QaLogFile.
6) Coverage Run macros and report an extremely rough measure of
command coverage in the file QaCoverage.
7) Cam Output This should be a button, not a selection in this combo
box. After macros have been run, Cam Output gives a means of
investigating the results of CAM macros that failed.
8) GDI Test Run macros and check for GDI leakage. Each macro is run
twice. The first run loads all of the necessary GDI objects. Any
additional GDI usage is reported as a leak in QaLogFile.
B) Run One Run a single macro (selected with file browser)
C) Run Directory Run all macros in a directory (selected with directory
browser)
D) Run All Run all macros in a directory and its sub-directories (selected
with directory browser)
E) Run List Run all macros listed in a user-specified macro play-list file
(selected with file browser). Each line of the play-list file is searched
for the pathname of a macro inside square brackets (e.g.
[Z:\mlv\mk589\Macro\InHouse\mlv\HistUndoVal.mac]). If the macro is not
specified by a full pathname with a drive identifier (e.g. "Z:"), it is
assumed that the path is relative to the directory that contains the user-

F)
G)
H)

I)
J)

K)

specified macro play-list file. Any line of the list that does not
contain a bracketed pathname is ignored. If a bracketed pathname is
preceded by text, it is ignored unless the text contains the word "Okay".
This allows you to use a "QaLogFile" as a macro play list, where only the
macros flagged "Okay" are executed. It is assumed the play-list file is a
standard ascii, new-line delimited text file.
Repeat Run the set of macros that were used in the previous macro run.
This does not remember the previous macro run after ZW3D has exited.
GDI Report Print extra information regarding usage of GDI objects in
QaLogFile. It is normal for the number of objects in use to increase as
more and more of the interface is used and brought into memory.
Memory Leak Generate an XML report of all the malloc memory leaks
encountered during the macros. The XML file is called vzw3d.exe.mem.xmlleaks and is written in the same directory that holds zw3d.exe. Use the
script MemLeak for a concise view of the leaks.
NOTE: THIS ONLY WORKS WITH A FULL DEBUG LINK AND ZW3D WILL QUIT WHEN THE
MACROS ARE FINISHED.
Memory Use Print total application memory usage information in
QaLogFile.
Start File When using Run Directory or Run All, this option lets you
start in the middle of the directory by picking a start file within the
chosen directory. This is especially useful when using a memory checking
tool such as Boundschecker that runs very slowly.
Pause On Error Pause a macro when it is failing so that the debugger can
be attached to diagnose the problem. Without this, the macro just fails
and execution continues with the next macro.

If there is a sub-directory named "Macro" in your ZW3D runtime directory,

"Run One" and "Run All" will bring up a file or directory browser initialized to
that directory.
Otherwise, the browser is initialized to the appropriate QA
macro directory on the source trees, based on the version of ZW3D you are
running.
It would be "Source\Macro".
Select the macro, or directory of
macros, you want to run, and ZW3D will start executing them.
ZW3D reports the pass/fail status of each macro to the ZW3D message window and
to "QaLogFile" in your runtime directory.
Macro-processing will generally
error out and be recorded as a failure if a ZW3D command fails, but this is not
always true.
In particular, many ZW3D commands simply log a warning message
when they receive bad input, rather than error out. Therefore, a QA macro
should extract some data and compare it to baseline data to determine whether
the macro succeeds or fails.
The comparison of output data to baseline data is
done automatically by ZW3D if the output data is logged with macro commands
supplied for that purpose.

ZW3D QA Commands
This section documents commands used to support macro-based QA testing.
------------------------------------------------------------------------

[QaAllowRetry]
Sets a flag that allows a QA Macro to use the Retry
command fails. Without setting this (the default is
and fail when any command fails. The value is reset
command. If a command fails during macro recording,
automatically inserted into the macro.

mechanism when the next

0), QA Macros will terminate
to 0 at the end of every
one of these commands is

------------------------------------------------------------------------

[QaOutBary,include_blanked,<point|symbol|assembly>]
Calculates a crude "bary center" from the curves associated with the active
target object (includes part edges). This point is logged to the output file.
The comparison of old to new is done based on the active proximity tolerance
(VgTolPoint) times ten.
If the text string "point" is included as the last argument to this command, a
"bary center" is calculated from the point entities in the active target object.
This includes up to 10000 points in each point block.
If the text string "symbol" is included as the last argument to this command, a
"bary center" is calculated from the non-curve geometric entities in the active
target object. This includes symbols, dimensions, text, annotation, hatch
patterns, and tables.
If the text string assembly is included as the last argument to this command,
a bary center is calculated from the curve entities in the target part and in
all of the components throughout the assembly.
The include_blanked flag determines whether or not the blanked entities within
the model should contribute to the bary center.
------------------------------------------------------------------------

[QaOutCAM,<multi>]
Writes CAM toolpaths.
"Multi" is an optional input. If specified, toolpaths for each CAM operation
are written to separate files. Each toolpath file is named
"macro_name.operation_name.bl" and is deposited in the same directory as the
parent QA macro.
When [QaOutCam] is replayed with the "multi" option, each regenerated toolpath
file is named "macro_name.operation_name.tmp". Each new ".tmp" file is compared
to the corresponding old ".bl" file using a simple "diff". If the "diff" fails,
the old and new files are compared line by line. If the line-by-line comparison

fails, the ".tmp" file is left in the QA macro directory so it can be "manually"
inspected and compared to the original ".bl" file. Otherwise, the ".tmp" file
is deleted.
------------------------------------------------------------------------

[QaOutComp]
Writes the component names and matrices of all the first level components in the
active part. For the rotation part of the matrix, the old and new values are
compared with a tolerance of 0.001. For the translation part of the matrix, the
old and new values are compared with ten times the active proximity tolerance
(VgTolPoint).
------------------------------------------------------------------------

[QaOutDbl,label,double_precision_value,<tolerance>]
Writes a double precision number and tolerance.
The comparison of old to new is done based on tolerance.
If no tolerance is specified, the comparison is based on V_ZERO.
------------------------------------------------------------------------

[QaOutDbl2,label,double_precision_value,<percentage>]
Writes a double precision number and percentage (0-100). The comparison of old
to new is based on a maximum percent difference. If a percentage is not
specified, one tenth of one percent is used.
------------------------------------------------------------------------

[QaOutDim]
Writes the names and values of all of the dimensions associated with the active
target. The comparison of old to new values uses a maximum difference of 1%.
------------------------------------------------------------------------

[QaOutDispAttr]
Writes the display attributes of the selected entity.
------------------------------------------------------------------------

[QaOutDist,label,double_precision_distance]
Writes a double precision distance (mm). The comparison of old to new is done
based on the active proximity tolerance (VgTolPoint).
------------------------------------------------------------------------

[QaOutEntCnt,ignore_blank,<layer_name>]
Writes entity counts for the active target object. If "ignore_blank=1", blanked
entities are not counted. <layer_name> is an optional input. If it is included
in the argument list, only entities on the specified layer are counted.
------------------------------------------------------------------------

[QaOutFaceCtr,face_index]
Calculates a "centroid" for the specified face based on the average of the
vertices of a triangle mesh that approximates the face to within ten times the
active proximity tolerance (VgTolPoint). This point is logged to the output
file. The comparison of old to new is based on the active proximity tolerance
(VgTolPoint) times ten.
------------------------------------------------------------------------

[QaOutHist]
Writes information about the history of the active target.
------------------------------------------------------------------------

[QaOutHoles]
Writes centerline coordinates of holes in the active part. The comparison of
old to new is based on the active proximity tolerance (VgTolPoint) times ten.
------------------------------------------------------------------------

[QaOutInt,label,integer_number]
Writes an integer to the active output file. The comparison of old to new is
"exact".
------------------------------------------------------------------------

[QaOutLayers]
Writes information about the layers.
------------------------------------------------------------------------

[QaOutLine,label,x1,y1,z1,x2,y2,z2]
Writes the given line start and end point coordinates (millimeters) to the
active output file. The comparison of old to new is based on the active
proximity tolerance (VgTolPoint) times ten.
------------------------------------------------------------------------

[QaOutLinks]
Writes the contents of the Link Manager associated with the active target
object. If the target object does not support the Link Manager, or there are no
links in the Link Manager, the text string "no links" is output.
------------------------------------------------------------------------

[QaOutPnt,label,x,y,z]
Writes the given point coordinates (millimeters). The comparison of old to new
is done based on the active proximity tolerance (VgTolPoint).
------------------------------------------------------------------------

[QaOutPnt10x,label,x,y,z]
Writes the given point coordinates (millimeters). The comparison of old to new
is done based on ten times the active proximity tolerance (VgTolPoint).
------------------------------------------------------------------------

[QaOutRoot]
Writes the name of the active file followed by the root object list
associated with the active file. Includes "invisible" root objects.
------------------------------------------------------------------------

[QaOutSeparator]
Writes a separator line.
------------------------------------------------------------------------

[QaOutText,arg1,arg2,...]
Same as the macro [Print] command, except that output is directed to the active
QA output instead of the ZW3D message area.
------------------------------------------------------------------------

[QaOutVar]
Writes the variables associated with the active target. The comparison of old to
new value is done based on a maximum percentage. By default, the test values
must be within 1% of the original value.
------------------------------------------------------------------------

[QaOutVol,<percentage>]

Writes the volumetric properties of the first shape (including inner shape)
associated with the active target part. If the shape is "closed", area, volume
and centroid are output. If the shape is "open", only area is output. The
comparison of old to new is done based on a maximum percent difference. If a
percentage is not specified, one tenth of one percent is used.
------------------------------------------------------------------------

[QaOutZoomCen]
Does a "zoom all" (without redraw) on the active view of the active object and
outputs the 2D centroid (in view plane space) of all the points evaluated during
the "zoom all. The comparison of old to new is based on the active proximity
tolerance times twenty.
------------------------------------------------------------------------

[QaPrint,arg1,arg2,...]
Same as the macro [Print] command, except that output is directedto the QA log
file (QaLogFile) instead of the ZW3D message area.
------------------------------------------------------------------------

[QaSuppFilePath,filename,pathname]
Gets the pathname of the specified support file (filename) based on the
directory that the active macro resides in.
------------------------------------------------------------------------

[QaUpdate]
Returns 1 if the QA macro should update its baseline files.
Otherwise, returns 0. (Macros do not normally need to bother with this.)
------------------------------------------------------------------------