root/design/testing/test_plans.tex@437

Revision 437, 37.3 KB (checked in by jelinson, 2 years ago)

added description of spec analysis tesT

Line
1\documentclass[pdftex,12pt,a4paper]{article}
2
3\usepackage[usenames,dvipsnames]{color}
4\usepackage[margin=1in]{geometry}
5\usepackage{verbatim}
6\usepackage{longtable}
7\usepackage{hyperref}
8
9\setlength{\parindent}{0pt}
10
11\newcommand{\tc}[2]{\textcolor{#1}{#2}}
12\newcommand{\hypt}[2]{\hypertarget{#1}{#2}}
14
15\begin{document}
16
17%%%%%%%%%%%%%
18\newcommand{\HRule}{\rule{\linewidth}{0.4mm}}
19\begin{center}
20\HRule \\
21\textsc{Erin Coughlan, Julius Elinson, Michael Morton, Rebecca Thomas}\\[.1cm]
22\textsc{\Large{Test Log \& Plan}}\\[-.1cm] % Title
23\HRule \\
24\end{center}
25%%%%%%%%%%%%%
26
27\part*{Test Results}
28Here we keep a hyerlinked log of the tests ran and the bugs found.
29\section*{Test Log}
30
31\begin{center}
32
33\begin{longtable}{| l  | l | l | p{5.5cm} | p{4.5cm} | }
34\hline
35ID & Date & Time & Action & Result \\
36\hline
37JE & 3/22/12 & 11:00 PM & Created directory and suite for adding unit tests & - \\
38\hline
39 JE & 3/22/12 & 11:30 PM & Added section in test plan for \hyperlink{spec:init}{\tc{blue}{Spec initialization}} and implemented it & Passed\\
40 \hline
41 JE & 3/23/12 & 2:15 PM & Wrote equality and print operators for Spec class to use in test & -\\
42 \hline
43 JE & 3/23/12 & 2:30 PM & Added test for \hypl{spec:get}{\tc{blue}{Spec getters}} & Passed\\
44 \hline
45 JE & 3/23/12 & 2:40 PM & Added test for \hypl{spec:set}{\tc{blue}{Spec setters}} & Passed\\
46 \hline
48 \hline
49  JE & 3/23/12 & 3:00 PM & Added test for \hypl{spec:sub}{\tc{blue}{Spec subtraction}} & Passed\\
50 \hline
51   JE & 3/23/12 & 3:15 PM & Added test for \hypl{state:build}{\tc{blue}{State building adding and deleting}} & Passed\\
52 \hline
53    JE & 3/23/12 & 3:30 PM & Added section in test plan for \hypl{state:research}{\tc{blue}{State research}} and implemented it & Failed, see the \hypl{bug:negres}{\tc{blue}{negative research bug}}\\
54 \hline
55     JE & 3/23/12 & 4:00 PM & Added section in test plan for \hypl{state:calc}{\tc{blue}{State turn change}} and implemented it & Passed\\
56 \hline
57 JE & 3/26/12 & 11:00 PM & Ran black box, system test for \hypl{main}{\tc{blue}{main.py}} as if actual user, clicking everywhere to test GUI responses &  Passed with bugs, see the \hypl{bug:button}{\tc{blue}{disappearing button bug}}\\
58 \hline
59 RT & 3/31/12 & 1:00 PM & Modified existing tests for better output & -\\
60 \hline
61 RT & 3/31/12 & 1:15 PM & Added test for \hypl{state:init}{\tc{blue}{State initialization}} and implemented it & Passed\\
62 \hline
63 RT & 3/31/12 & 2:00 PM & Discovered that tests were using incorrect building dictionary and modified to use full dictionary & - \\
64 \hline
65 RT & 3/31/12 & 4:30 PM & Added test for \hypl{state:check}{\tc{blue}{Check Thresholds}} & Passed\\
66 \hline
67 RT & 3/31/12 & 7:30 PM & Added building checks to  \hypl{state:check}{\tc{blue}{Check Thresholds}} & Passed\\
68 \hline
69 EC & 4/02/12 & 11:30 PM & Ran black box test for \hypl{top:save}{\tc{blue}{Save and Load}} as if actual user saving and loading games & Passes with bugs, see the \hypl{bug:loadFalse}{\tc{blue}{loading non-exisiting name bug}} and \hypl{bug:refreshBlock}{\tc{blue}{refreshing blocks bug}}\\
70 \hline
71 JE & 4/03/12 & 1:00 AM & Ran preliminary \hypl{scene:b/e}{\tc{blue}{begin/end test for Scene class}} &Passed with bugs, see \hypl{bug:skipinto}{\tc{blue}{skipping intro bug}}\\
72 \hline
73 EC & 4/06/12 & 3:30 PM & Updated test for \hypl{state:check}{\tc{blue}{Check Thresholds}} to meet more detailed return type & Passed\\
74 \hline
75 EC & 4/09/12 & 11:00 PM & Ran black box test for \hypl{map:endTurn}{\tc{blue}{end turn}} as if actual user continuously hitting end turn & Passed by bugs, see the \hypl{bug:outOfMemory}{\tc{blue}{end turn out of memory bug}}\\
76 \hline
77 EC & 4/09/12 & 11:00 PM & Ran black box test for the \hypl{tutorial:responses}{\tc{blue}{tutorial responses}} as if actual user moving through the tutorial & Passed with bugs, see \hypl{bug:skipTutorial}{\tc{blue}{skipping tutorial bug}}\\
78 \hline
79 JE & 4/10/12 & 2:00 AM & Ran white box test for the \hypl{specanal:to buy}{\tc{blue}{spec analysis suggestions}} & Passed\\
80 \hline
81\end{longtable}
82\end{center}
83\newpage
84\section*{Known Bugs}
85\begin{center}
86\begin{longtable}{| l  | l | l | p{5cm} | p{5cm} | }
87\hline
88ID & Date & Time & Description & Context \\
89\hline
90\hypt{bug:negres}{JE} & 3/23/12 & 4:00 PM & State erroneously allows more research to be performed than the current money, resulting in negative money. Moreover, it allows for negative research to be performed, which would provide a way to generated money. & This bug was found on the alpha release. It was found by running the Python unit test $$\verb+test_research()+$$ in the class$$\verb+/src/tests/test_state+$$ The assertion failures catch and describe the bug with a specified message.\\
91\hline
92\hypt{bug:button}{JE} & 3/26/12 & 11:00 PM & The right arrow button on the map display disappears & This bug was found on the March 26 prototype release. It was found by launching the game, then New Game $\rightarrow$ Build $\rightarrow$ Power $\rightarrow$ Windmill $\rightarrow$ Buy $\rightarrow$ End Turn. Then click on the icon of the windmill on the map and the right button disappears. Notice, however, that it disappears only for that block (which you can tell by selecting the icon of any of the other blocks)\\
93\hline
94\hypt{bug:loadFalse}{EC} & 4/02/12 & 11:30 PM & The game just creates a new game and starts there without providing an alert that the user was unable to load or giving the user a second opportunity to load a game. & This bug was found by attempting to load a game that does not exist, i.e., one that was not previously saved.\\
95\hline
96\hypt{bug:refreshBlock}{EC} & 4/02/12 & 11:30 PM & The map blocks do not reset to block 0 or whatever the correct block is for the saved game.  & This bug was found by starting a new game or loading a previously saved game after beginning progress on a new game and switching blocks.  The block that was open in the new game is the same which is open after these actions are preformed.  However, the menus are reset and any open ones are closed upon either of these actions. \\
97\hline
98\hypt{bug:skipinto}{JE} & 4/03/12 & 1:00 AM & The introduction scene can be skipped by starting a new game, going back and then resuming. Essentially, the scene can be truncated since resuming does not take the progression of the scene into consideration. & This bug was found in the most recent release of the game by doing: New Game $\rightarrow$ [Left arrow] $\rightarrow$ Resume. This will bring the player to the main screen, thereby skipping the remainder of the introduction scene.\\
99\hline
100\hypt{bug:outOfMemory}{EC} & 4/09/12 & 11:00 PM & The game runs out of memory by building one object and then continuously clicking end turn. & This bug was found in the April 10 Prototype by clicking New Game $\rightarrow$ Clicking through intro $\rightarrow$ Clicking through tutorial $\rightarrow$ End Turn (many times quickly).  Pygame crashes with error message Pygame.error Out of Memory.  Repeating this also gives various chains of updates where it crashes with the same error. \\
101\hline
102\hypt{bug:skipTutorial}{EC} & 4/09/12 & 11:00 PM & The tutorial can be skipped by using the left or right arrow keys to switch map blocks. & The bug was found in the April 10 Prototype by clicking New Game $\rightarrow$ Clicking through intro $\rightarrow$ [Left arrow] or [Right arrow].  This will bring the player to the main screen, skipping the entire tutorial.\\
103\hline
104\end{longtable}
105\end{center}
106\newpage
107\part*{Test Plans}
108\section*{Risk Analysis}
109One high risk area of our program is the State class because it handles the calculations for all of the game logic.  It has to determine which buildings can be built based on thresholds as well as calculate changes in the stats based on the Building that was built.  Other high risk areas include the Window and WindowEntry classes because everything that is displayed on the screen is part of a Window and each Window is made up of possibly several WindowEntry objects.  Since each WindowEntry is made up of a combination of images, rectangles, and text they are mostly unique and a bug that shows up in one might not show up in another.  Additionally, the WindowEntry class determines what response should be sent based on where a click occurs, which is a vital component of the user interface.  The Game class is high risk because it handles the parsing of responses and determines which methods need to be called to complete the desired task.  Furthermore, the Map and Block classes include a lot of hand-coded resizing and dynamic changes. Thus their implementation is risky and should be tested rigorously and together since they work intimately and exclusively together. Similarly, TurnMenu has high risk because it must dynamically compute what options should be available in a particular menu at run-time and then construct such a menu. Thus the reliance on this class to provide the necessary Window containing a requested menu exposes the program to the risk of the class's correctness in its computations.
110\section*{Tests}
111Note that the tests are listed in the following format:\vspace{-.5cm}
112\subsection*{Class}
113\begin{description}
114 \item[method1(arg)]-- Type of Test\hfill\\
115 Description
116  \item[method2(arg)]-- Type of Test\hfill\\
117 Description
118\end{description}
119\subsection*{main.py}
120\begin{description}
121 \item[\hypt{main}{main()}]-- Black Box, System Test via GUI\hfill\\
122 To test the system as a whole, we will continuously and relentlessly play our game. This will function to expose any errors that become apparent to a player over the course of extended use. We will explore new routes each time and test new features from the user standpoint as they are added. Essentially, we will test the entire software as a whole by approaching it like our players will and note the bugs that arise. Since every possible sequence of moves cannot be simulated, we will play extreme cases like only buying/researching in one category and technology, trying to do poorly in the game, and rigorously exploring new elements as they are added to the game.
123\end{description}
124\subsection*{Top}
125\begin{description}
126\item[\hypt{top:save}{saveGame(getName), loadGame(getName)}] -- Black Box, Regression Test \hfill\\
128\end{description}
129\subsection*{State}
130\begin{description}
131\item[\hypt{state:init}{\_\_init()\_\_}] -- White Box, Unit Test \hfill \\
132 The purpose of this test is to check that the state is initialized with the appropriate buildings and research values.
133 \item[\hypt{state:build}{addBuilding(building), deleteBuilding(building)}] -- White Box, Integration, Regression Test \hfill\\
134 The purpose of this test is to ensure that State correctly updates as it interfaces with the Specs and Building classes upon adding a building. This is a key component to the game logic as a player progresses. Moreover, it is complicated because it involves changes to States due to choices made in the most recent turn and choices made throughout the entire game. Since this ultimately keeps track of the player's game, we want to rigorously test its calculations and its integration with the other classes used to retain data. This will be a white box test because we will observe the arithmetic used in updating values to try to find boundary cases, such as negative values, in order to test the correctness of our implementation. This will be a scripted test where we will create a number of different States and simulate both different changes to the game and the effect of multiple turns elapsing, while at each step checking the current State with the expected State. This will be a regression test that we will run as new States or building types are added, ensuring that the method works as it should under such changes and that the game logic is robust regardless of other implementation changes.
135  \item[\hypt{state:research}{research(category, value)}] -- White Box, Integration, Regression Test \hfill\\
136 The purpose of this test is to ensure that State correctly updates as it interfaces with the Specs and Building classes upon performing research. This test is very similar in spirit to the \verb+addBuilding+ test, but since it receives a raw value representing how much to invest, it also tests more boundary values. In particular, it checks if the research exceeds the amount of money available, as well as if the amount to research is negative, which would allow the player to generate money.
137 \item[\hypt{state:check}{checkThresh(building)}] -- White Box, Integration, Regression Test \hfill \\
138 This test is designed to ensure that buildings only become available when their research thresholds are met.  This test checks that the stats are high enough, as well as research values and that required buildings exist.  This is  a white box test since we need access to the data members that store buildings and research values.  This will be a regression test that is used when we add new buildings as well as modify the requirements for existing buildings.
139 \item[\hypt{state:calc}{calcTurnChange()}] -- White Box, Integration, Regression Test \hfill \\
140 The purpose of this test is to ensure that State correct updates at the end of the turn based on the continuous Specs of its current Buildings. This reinforces the tests for Spec arithmetic, as well as ensures that the game logic will accurately compute changes to the player's progress at the end of each turn. The test will be crafted to the particular implementation to explore edge cases and hence will be a white box test. This will be a regression test that we will run as new States or building types are added, ensuring that the method works as it should under such changes and that the game logic is robust regardless of other implementation changes.
141  \item[displayStats()]-- Black Box, Component Test via GUI\hfill\\
142The purpose of this test is to ensure that the game data contained in Stats is properly displayed as a progress bar. Since this is all hand-coded using relative values to determine sizing of the rectangles representing the bars, the implementation is very risky and prone to bugs. Thus we will simulate multiple States and various changes to them and observe the results. Since the return value is simply a Window, the test must be done through the GUI and inspected for correctness.
143\end{description}
144
145
146\subsection*{Spec}
147\begin{description}
148\item [\hypertarget{spec:init}{\_\_init()\_\_}] --\\
149White Box, Unit Test\hfill\\
150The purpose of this test is to ensure that the initial starting values for a game Spec are properly initialized to the constants defined in a specific external file.
151 \item[\hypt{spec:get}{getMoney(), getPower(), getMaterial(), getSpace(), getPollution()}] --\\ Black Box, Unit Test\hfill\\
152 The purpose of testing these similar methods is to ensure that the Stats returned have the correct values.  These stats are the main component of tracking progress in the game, and ultimately determining if the player wins or loses.  We will test this method by creating various Specs using boundary case values (like negative and floating point values) as stats and making sure that each of these are returned correctly.  This is a black box, unit test because we are simply testing that the methods work as they should, without designing the tests around their implementation.
153  \item[\hypt{spec:set}{setMoney(newMoney), setPower(newPower), setMaterial(newMaterial)},]\hfill\\\textbf{setSpace(newSpace), setPollution(newPollution)} -- Black Box, Unit Test\hfill\\
154 The purpose of testing these similar methods is to ensure that we can change the stats in a given Spec correctly.  As before, these stats are the main means of tracking progress in the game.  We will test this method by creating various Specs using boundary case values for the Stats and then changing them.  We will also check to ensure that this method provides a deep copy and not a shallow one. Since the getStat() method would be tested before, we can call that method to make sure the Stats are correctly changed.  Therefore, this will be a black box unit test because we will write the test knowing the interface but not the implementation.
156 Note that these correspond to two similar, but separate tests. The purpose of testing these methods is to make sure that we can add and subtract Specs throughout the game.  This is integral of the game logic because at the end of every turn, these Specs are used to update the State of the game.  It is a data structure that needs to work correctly for the rest of the game to function properly.  We will test this by creating various Spec using boundary case values.  We will then add and subtract the Specs and check to ensure we get the expected result.  We also will look at adding or subtracting one Spec from itself to make sure that there are no problems in this regard.  This will be a black box unit test since we know that we can add and subtract, but do not need to know the implementation to design the test. \end{description}
157 \subsection*{SpecAnalysis}
158\begin{description}
160 The purpose of this test is to ensure that SpecAnalysis properly determines recommendations based on the provided state. This functionality is used in making recommendations to the player as they progress throughout the game, and thus checking that the suggestions are made correctly is an effort to ensure the educational message is both as intended and complete. Moreover, this test is an additional test between the interaction between Spec and State, providing an alternative means of testing these classes.
161\end{description}
162 \subsection*{Scene}
163\begin{description}
164 \item[\hypt{scene:b/e}{begin/end responses}] -- Black Box, Component Test\hfill\\
165 The purpose of testing this is to ensure that the continuity of a scene is not distributed by going backwards to before the start of the scene and then resuming the scene. Furthermore, we want to ensure that the end of scenes lead to the correct next screen.
166\end{description}
167\subsection*{Tutorial}
168\begin{description}
169\item[\hypt{tutorial:responses}{general responses}] -- Black Box, Component Test \hfill\\
170The purpose of testing this is to ensure that the player must progress through the tutorial by only clicking on appropriate buttons.  We want to make sure that these buttons respond as they would in the game and that clicking on other buttons or keys, there is no response.  This is a black box test unit test because the tutorial consists of other subclasses and we need to only visually ensure that nothing is being skipped or working incorrectly.
171\end{description}
172\subsection*{Window}
173\begin{description}
174 \item[response(pos)]-- White Box, Component Test\hfill\\
175The purpose of testing this method is to ensure that the default response for the Window object is returned when appropriate. This will be a white box test because we will design the tests knowing the Window object first checks its components for a response and only returns the default response when there were no collisions. Since this function takes coordinates and returns a signal, we can write a component test that creates various Window objects and then computes responses for various input positions and then checks for the expected response.
176  \item[response(pos)]-- White Box, Integration, Regression Test\hfill\\
177The purpose of this second test for the same method is to ensure that Window and WindowEntry integrate as they should. The testing procedure will be similar to the first test of this method, but will be targeted at checking for correctly translating coordinates and correctly accessing data members of WindowEntry from in Window. As before, this will be a white box test since the tests will consider the order in which responses are determined. Lastly, this will be a regression test because modifications to these classes and elsewhere should not affect the process for determining responses, which is an essential element to the operation of the GUI. Thus we will want to this framework to consistently operate as it should.
178 \item[render(surface)]-- Black Box, Integration, Regression Test via GUI\hfill\\
179As with the corresponding test for WindowEntry, this test will ensure that WindowEntry objects are rendered properly on top of each on a single surface. As this returns a visual result, we will have to test it by creating Windows with different WindowEntry objects and then observing that they render correctly. Since this is ultimately a test of our understanding of how PyGame renders on a surface, it will be a black box test, since the tests will not be specific to any method implementation.
180\end{description}
181
182
183
184
185\begin{comment}
186\subsection*{BuildingStats}
187\begin{description}
188 \item[getSpec()]-- Gray Box, Unit, Integration Test\hfill\\
189 The purpose of testing this method is to ensure that the continuous and initial Specs can be returned.  Thus, this is an integration test because it relies on the Spec class.  We will test this by creating Specs as in the Spec class tests and then making sure these same Specs are returned.  We will also test changing the Specs using the Spec class, and then trying to get the Spec.  This is a gray box unit test because we know that Specs are being used and that we can change these, but we do not know exactly how this method works.
190  \item[setSpec(newSpec)]-- Gray Box, Unit, Integration Test\hfill\\
191 The purpose of testing this method is to ensure that we can update Specs outside of the Spec class, by replacing the entire Spec.  We will test this method by creating various Specs as in the Spec class tests and then changing these values.  We will also test this by assigning one Spec to another and making sure each still updates and is displayed on their own.  Since the getSpec() method will be tested before this, we can use it to make sure the Specs are correctly changed as it would be used in the rest of the program.  Therefore, this is a gray box unit test as well as an integration test since we rely on the Spec class working correctly, and we do not know the exact implementation of the method.
192\end{description}
193\end{comment}
194
195
196\subsection*{Block \& Map}
197\begin{description}
198 \item[addBuilding(name)]-- Black Box, Integration Test\hfill\\
199 The purpose of testing this method is to ensure that any given building can be added to the map.  This is important because this is one of the main means of feedback and entertainment for the player.  We will test this by adding buildings to the map, including the special cases when we add  buildings that already exist as well as when we add those same buildings in different map Blocks. This is black box integration test as it tests the interface between Block and Map, but only using Map's interface to design the test.
200  \item[\hypt{map:endTurn}{endTurn()}]-- Black Box, Integration Test\hfill\\
201 The purpose of testing this method is to ensure that at the end of a turn, the map is updated with all the correct buildings that were made during one turn.  We will test this by creating a list of buildings that will need to be updated in the map.  To test some boundary cases, we will add multiples of the same buildings as well as add no buildings during a turn.  Because we do not need to know how buildings are being added in these tests, this is a black box test.
202 \item[checkSpace()]-- White Box, Unit/Integration Test\hfill\\
203 The purpose of testing this method is to ensure that we can correctly detect when there is space to build a building.  We will test this by asking to add buildings one at a time, including when there is no more room on a given screen.  We will make sure that the buildings are only allowed to be added when there is space.  Since we know what the max number of buildings on a screen is, this is a white box test.  It is a unit test because the information depends only on this method and nothing else, while still being an integration test because it involves the interface between Map and Block.
204\end{description}
205
206
207%% Deprecated tests based off previous architecture
208
209\begin{comment}
210\subsection*{Deprecated Classes as of March 26 Prototype}
211
212\subsection*{Game}
213\begin{description}
214 \item[endTurn()]-- White Box, Integration Test\hfill\\
215 The purpose of testing this method is to ensure that the game can be updated at the end of every turn.  This is extremely important because this method interfaces between several other classes in order to update every aspect of the game. Thus we will test this higher-level method correctly calls on the lower-level methods to end a turn in the correct order and that there are no unexpected side effects. We will test this method for turns that include multiple actions and no actions as boundary cases. Furthermore, one of the risks associated with this function is that it transfers data from one State to another. Since this changes critical information, we will want to ensure this function works properly and will design the test by examining how this operation is performed.
216  \item[createWindow(name)]--White Box, Component Test\hfill\\
217 The purpose of testing the createWindow method is to ensure that the correct Windows are created when prompted.  This is important because this information will be used by main to create a list of the current Windows, which are displayed in the UI class.  Since we know that the list is split on the underscore character, we will test this method by giving it a sequence that involves many underscores in a row as well as sequences that have no underscores.  This is a white box test because we know exactly how the information is being parsed.  It is a component test because the parsing is the base of many other classes as well as this function encompasses one of the main jobs of the Game class.
218 \item[build(name)]-- Black Box, Integration Test\hfill\\
219 The purpose of testing the build method is to ensure that the buildings the user decides to create are indeed being created, that there is space for the building to be created, and that the building actually can be created based on the current game progress.  Because of this task, the build method calls upon other classes, such as Map and Turn, and therefore this will be an integration test.  We will test that this method correctly calls the other classes' methods without side effects and that the overall response is that a new building is created, as desired.  We will try edge cases, such as creating buildings when there is have no space in the current block, insufficient stats in the State, and building something has not yet been unlocked based on the game progress.  This will be a black box test as we will examine Game's interface without looking at its implementation in order to test these situations.
220\end{description}
221
223\begin{description}
224 \item[calculateBuildables(State)]-- Black box, Integration Test\hfill\\
225 The purpose of this test is to ensure that TurnMenu can properly interface with State, as well as Building, to correctly determine what can and cannot be built by the player at a given moment. This can be tested by passing the method a number of different States and compare the returned result with expected values of what is and isn't buildable. This is important because menu options change as the game proceeds and we want to ensure we will be able to correctly accommodate those changes. This will be a black box test because we will simply design the tests around the method interface, without examining the implementation.
226  \item[createBuildMenu(State)]-- White Box, Component Test via GUI\hfill\\
227 The build menu is dynamically created and thus this function constructs the appropriate Window to return for a given State. A major concern is whether the Window will have overflow if there are a lot of things that are buildable. This will be a white box test, as it will examine the for loop that is used to create the menu and try to cause an overflow. It will also test the ability to retrieve the necessary information and image for the menu items during run time as soon as the menu is opened. Since the return result is a Window, we will have to evaluate the results of the test with the GUI.
228\end{description}
229
230\subsection*{WindowEntry}
231\begin{description}
232 \item[isCollision(pos)]-- White Box, Unit, Regression Test\hfill\\
233The purpose of testing this method is to ensure the correctness of the technique to determine if a collision has occurred with a WindowEntry, which is essential for parsing graphic user interface input. However, because the parameters are just a position, we can test this method by constructing a WindowEntry with an assortment of different rectangles and images, and test the collision response for a number of coordinates. We will test boundary cases like negative coordinates, empty WindowEntry objects, etc. Thus it will be a simple unit test of this specific method and will be a white box because we will design the tests knowing the order in which the WindowEntry's individual components are tested for collisions. Furthermore, this is a regression test because we want to ensure this functionality remains intact as we develop elsewhere and possibly add to the contents of WindowEntry.
234\item[render(surface)]-- White Box, Component, Regession Test  via GUI\hfill\\
235The purpose of testing this method is to ensure that elements of a WindowEntry are rendered properly onto a surface and in the correct order. Since the return type is a surface, the test requires visual inspection to verify correctness through the GUI. We will construct WindowEntry objects with an assortment of overlapping components, added in different orders, and verify how they are rendered. This will be a white box test because we will design the tests cognizant of the order in which a WindowEntry's components are rendered to the surface. Furthermore, this is a regression test since this is how all visuals are rendered and we want to ensure this remains correct throughout development.
236\end{description}
237
238
239\subsection*{UI}
240\begin{description}
241 \item[eventCheck()]-- Black Box, System, Regression Test via GUI\hfill\\
242 The purpose of testing this method is to ensure that the user interface is working correctly and can recognize the user's mouse clicks.  We need to make sure that the clicks that are recognized have the same responses that the user expects.  If this method does not work correctly, the game will not function properly and the user will become frustrated, which will decrease the learning opportunities.  Knowing this, we plan on testing this method by clicking along the boundaries of buttons to make sure that no clicks are wrongly interpreted.   We will also make sure that each button has the proper response when clicked on.  Because this method is the primary means of interfacing with the user, this will be a regression test used throughout any GUI test.  Since the results will be visual, this is a black box test that will be done using GUI.
243\end{description}
244\section*{Rationale}
245As each individual test offers its own rationale, we will discuss the cumulative adequacy of the entire test plan based on the different types proposed. \\ \\
246We propose unit tests for WindowEntry, Spec, and Map \& Block. These are the primary low-level classes and retain actual game data. Thus by testing all of these with unit tests, we ensure the foundation of our program and provide a framework to extend beyond these classes in testing higher level classes, some of which must interface with these classes. Therefore these unit tests would allow us to proceed in checking more complicated classes and methods, knowing the simple components work as they should.\\ \\
247We propose component tests for WindowEntry, Window, TurnMenu, State and Game. All of these classes contain several methods that must be tested in tandem in order to fully test the class. Moreover, these classes all retain game data and component tests will ensure that the data is handled and manipulated properly through the various methods that they perform. Ensuring the accuracy of the internal game data is an essential requirement that will be addressed by these component tests.\\ \\
248We propose integration tests for Window, TurnMenu, State, Game and UI. These are the primary classes that have to work intimately with the other classes. In particular, all of these classes must interface with each other at some level. Thus we intentionally build in some redundancy in our testing by designing tests that target the interfacing performed in each of these classes. Moreover, the other classes not included here, like WindowEntry and Spec, do not access data from other classes, but instead are low-level. Thus integration tests for these classes that manage and interact with various other classes should be sufficient and thorough.\\ \\
249We propose system tests for UI and the entire program via main.py. A system test for our program would necessarily entail testing all the components by playing the game. We will use the game, paying attention both to the correctness of the UI responding to input and the general correctness of game logic and displays. By intentionally playing the game by looking for edge cases, focusing on new features and playing often, we will expose bugs that are apparent to the player, which are the types of bugs that must be infrequent as possible. Essentially, by experiencing the program as any user would, we will ensure that the game functions cohesively and as it should, testing the complete framework of the system.\\ \\
250Regression tests are proposed for WindowEntry, Window, State, and UI. The reason for testing these classes with regression tests is that these are the core classes that retain game data and dictate the program behavior. Thus as we add features and extend the program, these classes should retain entirely functional and unaffected. In other words, these are the classes that should work independently of changes elsewhere and should scale as the project and game does. Moreover, an error in these classes would cause an error in most other classes. Thus by using regression tests on these classes, we will isolate any bugs that do exist more effectively.\\ \\
251We propose both white and black box testing throughout. Any tests that require visual results to confirm the correctness of classes or methods was cast as a black box test since it will be handled through the GUI, which doesn't allow insight into the implementation. Moreover, simple methods that do not contain particularly complicated implementations can be tested via black box tests by only looking at the class interface. These tests simply ensure that the desired results are provided for particular methods and don't try to break a messy implementation. White box tests, however, address methods that are more complicated and contain more control statements. These tests were proposed for any methods with tricky implementations and for methods where the ordering of certain operations matters (like in what order things are rendered or checked for collisions). Moreover, white box tests were proposed for things that seem susceptible of working improperly (like TurnMenu that dynamically builds the menu with the chance for overflow).\\ \\
252We deem our test coverage thorough. We have proposed tests for all non-trivial classes in our program and for each nontrivial method. Moreover, various integration tests will test the same classes and methods multiple times. Thus each relatively risky class is addressed with tests both focusing on that class and via integration tests, as well as indirectly through system tests. This threefold approach to testing each essential to the game will ensure the coverage has both depth and breadth.\\ \\
253Some shortcomings of our test plan include a heavy reliance on GUI tests. These tests are cumbersome, require a user to walk through and do not provide clear, quantitative results. However, since this will be how the user experiences the program, it is important that we test from this perspective. Another shortcoming of our proposed tests is that Block and Map are tested together and thus a bug would require first determining in which class the problem is in. However, the reason for this coupling is that the classes work intimately together and the tests would be too contrived and unhelpful to test these classes in isolation.  Lastly, our test plans offered no system test that doesn't use the GUI. It was not clear how a test that did not require the GUI could be designed since in order to test the entire framework, since we rely on graphical user input for game events to occur. Nevertheless, it would be preferred if we could run an automated test that would check essential operations in the system as a baseline for the program's correctness.
254\end{comment}
255\end{document}
Note: See TracBrowser for help on using the browser.