root/releases/v1/doc/final_report.tex @ 683

Revision 683, 44.2 KB (checked in by jelinson, 2 years ago)

revised testing section of final report

6\newcommand{\status}[1]{\textsc{Status: #1}}
10\HRule \\
11\textsc{Erin Coughlan, Julius Elinson, Michael Morton, Rebecca Thomas }\\[.1cm]
12\textsc{\Large{Final Report - Michigan Technology}}\\[-.1cm]
13\HRule \\[.4cm]
16The problem this game seeks to address is the uninformed position many individuals of the current generation have in regards to the effects of technology. To solve this problem, the game that has been produced and delivered, TechTrek, seeks to educate its users on the impact of various forms of technology in a fun and engaging manner. In TechTrek, the player takes the role of an explorer who crashes on an alien planet, much like Earth, and must introduce technology to the natives so that they can help repair the spaceship, at which point the player can fly home. At the start of each turn the user will be given a choice whether to build or research in types of power, transportation or materials. The player will need to balance several stats, such as pollution, energy consumption, and space use, while furthering the planet's technology and development. Thus the player is awarded total control, but they must make careful choices, so that their actions do not lead to the decline of the planet. Preliminary results indicate that users enjoy the gameplay and quickly develop strategies, thereby demonstrating an understanding of the game dynamics and the intended message.
17\section*{Background and Problem}
18With the increasing presence of technology and its cascading effects, it is important for all generations to understand the impact this has on society. In particular, members of younger generations, who were born into the present state of technological excess, may not fully appreciate nor realize the large impact of technology. The problem thus becomes, as stated in the provided learning objective, ``Students should be able to explain how communities are affected positively or negatively by changes in technology." In other words, the problem which the game seeks to address is the disconnect between introducing certain technologies in communities and their impact. In particular, the students targeted by this learning objective are a group of middle school students from Kalamazoo, Michigan, although the message will be universal. The game was designed to educate its users about several specific modern instances of technology. These intended lessons are conveyed by rewards and penalties throughout the game as decisions about using certain technologies are made by the user. In short, the game addresses the problem of people not understanding the full consequences of introducing technology in society by empowering the player to make decisions about such and require that these decision be prudent and sustainable in order for them to be able to successfully complete the game.\\ \\
19The proposed game is not without competitors that possess similar objectives and target the same age group. As such, a study was conducted examining four of these games in order to assess their strengths and weaknesses, which were in turn used to refine the design and goals of the game as it was being developed. The games tested were Virtual City Playground, My Country, Civilization 2, and Village Defense \cite{vcp}\cite{mc}\cite{c2}\cite{vd}. A number of observations were made about these various and diverse games, especially regarding the effectiveness of relaying an educational message. Perhaps the most significant aspect observed is that games without clear endings fail to motivate the player to continue the game or to make prudent decisions. Thus in order to communicate the desired message to our players, the game must emphasize a goal that can only be obtained by making sensible decisions regarding the use of technology.\\ \\
20As stated, the audience for this game is specifically middle school students in Kalamazoo, Michigan. With this in mind, the problem is further refined by the context in which the game will be used. In particular, the targeted group of middle school students has discussed some aspects of the effects of technology on society and thus it is important that the game connect to these topics and feel accessible. The curriculum covered has included discussions of pollution, invasive species, and habitat loss. This provides a narrower context for the broad issue of technology. Another component of the context to this problem is that many of the students targeted by this game are from lower socioeconomic families and have below grade-level skills, particularly in reading. The relevance of these facts is twofold. On one hand, by relaying an important message about the use of technology to disenfranchised people, the game will educate a community that is often unreached by such messages and might not otherwise be alert to such issues. In addition, lessons about technology will serve to excite students about learning and reinforce basic reading and arithmetic skills during gameplay. \\ \\
21The necessity of a solution to the stated problem is made obvious by the increasing number of pressing issues that modern society faces as a result of spreading technology. Issues of pollution and environmental loss and destruction, although long-standing, are ever-relevant in current times. Educating students on these matters from an early age is an essential component to informing the public as a whole and building a sustainable future. Moreover, fostering a better educational environment for disenfranchised students is of vital importance to improve their learning experiences and the infrastructure of education as a whole.\\ \\
22Educational research suggests a number of potential impacts our solution offered through a game will have. A game about technology achieves two simultaneous effects: the direct lessons about technology conveyed through the game content and the meta-lesson offered by using computer game technology itself as a tool in the classroom. Thus addressing the problem through a computer game is a vital aspect of the solution. A survey of data compiled by the Milken Exchange on Education Technology indicated that students learn more through the use of computers and exhibit more positive attitudes towards school in general \cite{milken}. General computer use will also foster basic technical skills and may stimulate interest in the field. Computer literacy is becoming increasingly essential and a study by the Office of Educational Research and Improvement in the Department of Education confirms the success of instilling these skills in students who get exposure to technology in the classroom \cite{dept}. Additionally, a greater understanding about technology will make students more informed citizens and conscientious of the technology they use and the technology around them. In summary, the impact of the solution will ideally include a greater awareness of societal impacts, greater familiarity with, and interest in, computers, and improved perspectives and performance in school settings.\\ \\
23Given the stated problem, at the onset of designing the solution, a list of promised deliverables were generated as follows and are here annotated to reflect their status in our final deliverable:
25 \item A zipped directory containing the game executable and all necessary accompanying program resource and library files, ensuring that the program can be run on a Windows 7 netbook without any additional installations. From the time of download, the game will be playable within 1 minute. \status{Delivered}
26 \item A user's guide containing the following:
28 \item Installation guide specifically for installing on the machines that will be used in the classroom. \status{Delivered}
29 \item Gameplay instructions, describing game story and objectives, winning and losing, turn procedure and basic strategies. \status{Delivered}
30 \item Chart summarizing all the possible technologies a player can build, listing all stats for each that affect gameplay (e.g., cost, pollution effect, etc.) \status{Pending}
31 \item Troubleshooting section for anticipated technical issues \status{Delivered}
32 \item Known issues with software \status{Delivered}
35Thus upon revisiting the initial promised deliverables, the majority have been produced to specification and satisfaction. While we have produced a chart with the details of the game, since the specific value of the stats related to the buildings and other aspects would perhaps benefit from further refining, we do not consider this complete, though it has been produced.
36\section*{Requirements Analysis}
37To guide the design process, an enumerated list of requirements was generated at the onset of the project. In particular, both functional and non-functional requirements were established and prioritized individually. Functional requirements establish tangible properties of the deliverables, while non-functional requirements express more subjective, qualitative characteristics. The design process used this prioritization to weigh decisions throughout. The requirements here have been annotated with their status in our final delivered result.
38\subsection*{Functional Requirements}
40\item The game must be able to be installed, launched and played on the targeted hardware. In other words, it is our foremost functional priority that the game work as an executable program and not crash. \status{Met}
41\item The game must accurately portray at least three types of modern technologies. It must depict a number of benefits and costs to them in a way that develops gameplay. At its core, the game seeks to be educational and thus there must be realistic consequences portrayed. In particular, the game must depict in some way at least two of the following three topics: pollution, deforestation and invasive species. \status{Met}
42\item The game must have a clear and simple storyline that leads towards a specific end goal. This will motivate students to complete the game, ensuring that the educational goal is achieved. \status{Met}
43\item All buttons in the game should be clearly labeled and fully functional. There must be no ambiguity or nonoperational buttons, as this would severely hamper the user's experience with the game. Any incomplete or broken features would discourage students from completing the game, which would not allow them to learn all that the game could teach them. \status{Met}
44\item The text in the game must be simple and aimed at the middle school students for which the game has been created. The reading level for the bulk of the text must therefore be no higher than second or third grade, and any text that is more complex must not be integral to the basic learning objectives or playability of the game. \status{Met}
45\item The game must be able to be played in its entirety without crashing or stopping for any reason. Any significant bugs that prevent completing the game interfere with the learning objective. \status{Met}
46\item The player must be able to save their progress and then resume their play. This will ensure that a player can complete the game, even if it takes longer than a single class period. It also allows the player to try a couple of decisions and then revert to a saved checkpoint if they determine that they want to undo certain decisions. \status{Met}
47\item The player must be empowered to make decisions which determine their success in the game. By placing the decisions in the student's hands, they are more likely to personally understand the powerful impact that technologies can have on a community, as they will be the ones imposing such consequences. \status{Met}
48\item There must be metrics available to the player to gauge how well they are doing. These stats would help to focus the competitive nature of the students, which would in turn encourage them to continue playing the game. Moreover, as they will be updated with each move, they will serve as instant feedback for some of the bad decisions. \status{Met}
49\item The player should be able to win, even with several ``bad'' decisions. There must be a balance between making the game a challenge so the students don't get bored, and making the game not too difficult so the student will not become discouraged. If one bad decision made the game unbeatable, the game would require a specific series of moves and would simply become frustrating for the students to play. Instead, there will be many routes one can take to win, allowing imprudent decisions to be redeemed for more sensible ones. \status{Pending}
50\item The game should feature basic tutorials in the beginning to explain how the game works. \status{Met}
51\item Basic explanations of the technology should be available as players make decisions to use them or not, allowing them to discover why the different technologies are harmful or beneficial. This will both educate students on basic forms of technology and provide an explanation of why certain technologies have the consequences that they do. \status{Met}
52\item In addition to the stats available, there must be regular accomplishments awarded to the player for gratification throughout. This increases the enjoyment of the game and increases the likelihood that the students would play the game to completion. \status{Met}
54\subsection*{Non-Functional requirements}
56\item The most important requirement of our game is to achieve our educational goal. Thus, the game must convey its intended message through the small decisions the player makes. Ultimately, winning should coincide with the player having exercised good, sustainable decision-making regarding the use of an assortment of different technologies. \status{Met}
57\item The game must feel personal to the player through the use of characters and basic story. This will help to ensure that the students will learn as much as possible from the game, as they will be engaged with it. \status{Met}
58\item The game must be fun in order to make the user experience enjoyable and to better relay the intended lessons, as students will be more perceptible if they simply like the game. \status{Met}
59\item The game must feel intuitive and easy to learn. The mechanics of the game should not interfere with the gameplay itself and thus not interfere with the learning experience. In addition, as the targeted audience is a group of middle school students who will play the game over the course of one or several class periods, a player must be able to play through the game without the assistance or help of anyone. It should be clear how each turn works and what is required to win. \status{Pending}
60\item The game's framework should be easily understandable and extendible. The promised deliverable will provide many ways to expand and extend the game, including adding additional technologies. Moreover, this game may be revisited by summer research students, in which case the readability of the code will greatly affect the speed with which they can learn the framework, as well as how thoroughly they can develop and improve it. \status{Met}
61\item The game must appeal to a variety of students whose backgrounds with video and computer games might be significantly different. Thus, it must be accessible to students who have never played a game before, while still being interesting to those who have been playing for years. \status{Met}
62\item The quality of the artwork should feel consistent throughout the game and should be roughly on the same caliber of many games developed for mobile devices, to use one metric. \status{Pending}
64Regarding the pending requirements, these requirements are either considered only partially complete or have not been tested. In particular, the functional requirement \#10 would benefit from the building stats being optimized through a means other than trial and error. The non-functional requirement \#5 has not been thoroughly tested. Initial tests indicated players experienced some confusion with the gameplay and we have not had the opportunity to further test this aspect after making a number of revisions. Lastly, the non-functional requirement \#7, the consistency of the artwork, could be further improved by using entirely our own original artwork, ensuring consistency of style. However, time-constraints and talent-constraints limited our ability to develop \emph{all} of the game's artwork. \\ \\
65Given the extensive requirements, associated risks were established and planned for accordingly. The largest risk faced in the development of this project was the complexity and size of the game. Essentially the risk was whether or not there will be sufficient time to complete the game as it has been envisioned. Given this, development was focused on implementing the various portions of the game (beginning, turn sequence, ending), rather than first filling in all the details for a given section. Additionally, an intuitive game engine called Pygame was chosen to ensure streamlined development and avoid cumbersome debugging, as it has a simple application programming interface. We deem that we fully addressed this risk, as indicated by the scope and size of the produced game.\\ \\
66Another risk was the game's dependence on large amounts of artwork. The game was designed around avoiding large blocks of text, as the students that the game is targeted for have a low average reading level. This required that the game communicate messages and information through the use of clear images. Creating or finding all these images was be one of the most time-consuming portions of our project. Given this expected challenge, a decision was made to use freely-available, royalty-free vector graphics libraries, which are abundant on the internet. This risk, while addressed, remained an issue up through the final deliverable. Integrating artwork from different sources was difficult and could be further improved. \\ \\
67Another risk posed to the project was being able to elegantly handle user input through the Pygame engine and interface that with the game itself. This required an efficient means of sending information back and forth in a extensible way that avoids global variables and other bad programming conventions. To address this, all major of initial development was oriented around establishing a scalable framework to handle this task. It underwent many revisions throughout the agile design process and the final resulting framework used in the game has proved both scalable and elegant. It fosters simple feature development and presents a clean separation between the graphic user interface and the game logic.\\ \\
68The final risk the project faced was that it may prove ultimately not enjoyable and not fun for its targeted users. It is difficult to accurately view a game from a middle school student's perspective and thus there is a variable factor of actual pleasure the game will provide. Given this, the software is designed to allow changes in artwork, story details and available technologies, to accommodate any feedback we receive. By building a flexible framework, any changes in these details should be easy to accommodate, and thus the game can cater to the wishes of its audience. Thus while the final version of the game has not had user testing, if this were to happen, the game could easily accommodate suggested changes without changing the game architecture.
71\section*{Game Design}
72\subsection*{Game Story}
73The hereby delivered result, TechTrek follows the story of an intrepid space traveler whose spaceship crash lands on an unknown alien planet. The traveler is greeted by friendly aliens who are impressed by the spaceship's technology, which far surpasses the technology on their planet. The game's plot thus becomes the traveler trying to repair the spaceship and return home but only through first teaching the aliens about the various scientific achievements of his planet. Only when the aliens have achieved sufficient technological advancement will they be able to help repair the spaceship so that our hero can make the voyage back to their home planet.\\ \\
74The conflict of the story comes from the inherent difficulties of developing new technology. As the aliens have not yet been introduced to many of the necessary inventions, there will be many and possibly negative consequences of introducing them into the society. As the traveler knows from their own experience on Earth, gadgets like windmills, nuclear plants, airplanes, mines, etc., all can have a range of effects on society, both beneficial and harmful. Should the consequences of what the traveler chooses to show the aliens be too severe, the planet and aliens will suffer, which will prevent the traveler from getting back home. Thus the traveler must strategize on how they will  both achieve maximum technological advancement as well as provide the greatest prosperity to the aliens, while avoiding bringing about catastrophes. \\ \\
75Navigating all of these complex goals creates a game that requires planning, making prudent decisions beyond what benefits just the traveler, as well as responding to feedback. This story was selected because we sought to convey a society to the users where the expectation and assumption about the existence and convenience of modern technology is removed (hence the underdeveloped alien planet). Such a situation then allows the user to critically think about each and every technology they choose to introduce, rather than accept them as necessary and inevitable. Instead, the player can make their own decisions about whether it is worthwhile to use certain types of our current technology. This then fosters the learning objective by requiring players to study and scrutinize the effects of the various technologies in order to make informed decisions. Moreover, as will be described, there exist a number of feedback mechanisms that allow the user to reassess their previous decisions and make revisions to their strategy accordingly. This furthers the learning component and allows students to refine their understanding of how technology impacts society.
76\subsection*{Game Play -- Frontend}
77\subsubsection*{New Game}
78When a new game begins, the player progresses through an opening scene which explains the game story with both picture and text. One of the frames from the scene is shown in Figure (XX). At any point, the player can proceed through or go backwards in the scene, ensuring that the player can go at their appropriate pace.\\ \\
79After the opening scene is a walkthrough tutorial. All buttons on the main screen are inactive except for the button indicated by the current step in the tutorial. An alien on the screen guides the  player through the basic steps involved in the game, as will be described in the next section. This feature introduces the players to the game and ensures that they understand both the game play and the screen layout. This streamlines all subsequent turns and allows the player to focus on the game content, instead of the mechanics. One step in the tutorial is shown in Figure (XX).
80\subsubsection*{Each Turn}
81The game play is divided into turns. In a given turn, a player can do a number of things. They can select among the various available items to build; they can research in different fields of technology; they can search through the planet by toggling through the map; lastly, they can interact with the aliens, learn about and even undo their previous purchases. After the player makes all their decisions for the given turn, they commit these changes. This updates the player's progress and depending on that, an event may occur reflecting either that turn's changes or changes triggered by certain heuristics. For instance, if the player built something during the turn, a scene depicting the construction might occur. Alternatively, the game is programmed with heuristics that trigger events based on certain states in the game. Thus if the player builds multiple nuclear plants, they increase their probability of a nuclear meltdown. Such an event would take form in both a scene depicting the events and change in the player's progress.\\ \\
82These events are another form of feedback for the players. They reflect, with loose realism, likely events that would occur given certain decisions and thereby will influence the player's subsequent decisions and relay the relevant message to them. Moreover, these cut-aways and nondeterministic features keep the game play dynamic and lively. Thus even if the player attempts to recreate a game entirely, these events will ensure the outcome will not be identical, thereby keeping the game fresh and enjoyable for even experienced players.
83\subsubsection*{Game Logic and Calculations}
84Throughout the game, progress is measured in the form of 5 resources: money, food, power, material and pollution. Each decision the player might affect any subset of these. When a player goes to build a piece of technology, they must first have a sufficient amount of resources to pay for the item's one-time cost. If purchases, these costs will be deducted from the player's resources. However, as many technologies have lasting effects, items also have on-going costs, which might differ from the one-time costs, that are charged each turn. However, these might also be benefits, such as a windmill's production of energy. \\ \\
85In addition to resources, there are other thresholds that are required to purchase a item. For one, some items require having already purchases some other items. This requirement is in place to both reflect the reality of interdependence between technologies, as well as prevent players from finding shortcuts that bypass the essence of the game. Furthermore, buildings require a certain amount of research has been conducted. Thus a player can choose to perform research on a given turn in order to further advance development in that field and make new technologies available, as well as improve the resource-use of that field in general. This element of the game was added to convey the important development that is essential in producing new technologies.\\ \\
86Some events triggered throughout the game after each turn affect the player's resources. For instance, if an excessive amount of transportation introduces invasive species, the player's food supply drops.\\ \\
87Thus resource management is at the heart of both game logic and the player's strategy. They provide a quantity metric for both progress and an item's affect on society. Of course, these purely numeric indicators work in tandem the more visceral elements, such as events, alien feedback and events. \\ \\
88Alien feedback is determined by the items in their proximity. Nondeterministically, aliens respond either positively or negatively to their surroundings in order to convey the various perspectives an entire society might feel toward certain technologies. Additionally, feedback is given for each resource the player manages, with suggestions about ways to improve it, by either new purchasing or removing existing ones. All this feedback is produced dynamically based on the current state of the game. This keeps the game both interesting and as informative and educational as possible, by adapting feedback to the player's choices throughout the game.
90\subsection*{Screen Layout}
91The screen layout emphasizes consistency as well as engagement with the alien planet. As seen in Figure (XX), the main screen has a metallic frame that contains the essential buttons a player needs to interact with throughout the game. Except for scenes, this frame is always present and thus achieves a consistent user interface design that the user quickly becomes familiar with and used to. The toolbar is sleek and has only the essential elements, while menus that stem off of it allow the player to access all the elements of the game. Likewise, the player's resources are visible in the upper right hand corner and thus provide a clear reminder of the need to manage them responsibly.\\ \\
92The landscape occupies most of the screen. This is to provide the greatest amount of engagement with the alien planet for the players, both in seeing the terrain and interacting with the aliens. It also occupies most of the screen in order to convey a nearly first person experience.\\ \\
93The game supports hover events, which allow brief descriptions of the various graphical elements to be displayed instantly. The following are all types of information conveyed through the various hoverable elements: explanations of what a button does, an item's description, suggestions for how to improve a particular resource and reminders of the long-term goal.
94\section*{Game Design -- Backend}
96The game was developed in Python 2.7 using the game library Pygame 1.9. Using Pygame's API, we designed an otherwise from scratch graphic user interface to handle all graphics management throughout the gameplay. The artwork consisted of both original designs and composites of royalty-free vector graphics.
98The game architecture separates the game logic from the graphical elements. The program design is best conveyed in the domain diagram as seen in Figure (XX). This division allows for the greatest abstraction and reduces coupling. Earlier designs suffered from much more integration of these components, which proved to be not conducive to feature development, as it required making changes in both domains. \\ \\
99Every graphical element subclasses from the Window class and is designed based on the intuitive elements of the game screen. Each subclass specifies the way it handles events and has its own set of responses, as well as update routine. The game logic uses basic data-oriented classes to keep track of the game's state. Moreover, analyzers are used for both the state and resource analysis, in order to offer dynamic suggestions and feedback through the game. These are programmed with heuristics that we've created to provide responses given certain thresholds. Furthermore, because these heuristics are encoded in auxiliary files, the framework is very flexible to changing them, which makes the design extensible and easy to add content to the game.
103TechTrek, the presently-deliverable result, follows the story of an intrepid explorer whose spaceship crashes on an unknown planet.  Attempting to get home, the explorer must work with the planet's aliens. Unfortunately, the current technology of the aliens is not advanced enough to repair the ship quite yet. Thus the player must introduce various forms of technology to the aliens while bearing in mind the various impacts these technologies will have on the planet. These technologies range from material mining technique to trains and solar panels. In particular, the explorer will need to focus on balancing pollution, space use, money, supply of materials and energy consumption.  The game ends when the aliens have achieved sufficient technological advancement to allow the player to build a spaceship and return home. Alternatively, if the technology that is introduced disrupts the planet too greatly and the people grow too unhappy, the player loses as the planet deteriorates for one of many reasons.
105The back story is conveyed through a sequence of scenes depicting the flight and crash of the player's spaceship. The game begins when the player has been put in charge of developing technology for the planet. A tutorial guides the player during their first turn as they research in an area of technology and then build something. The tutorial features explanatory text as well as arrows leading the user to the correct button. Moreover, all other buttons are disabled during this so that they must learn to walkthrough these steps, which are imperative to the overall gameplay.
107For subsequent turns, the player is presented with the option of either researching or building various technologies. In order to be able to build certain, more advanced forms of technology, the player must first invest money in the education and development of that form of technology. Only after the money invested exceeds a predetermined threshold will the player be able to unlock and then build the technology. To aid in the learning process, when the user clicks on a specific piece of technology, a screen containing basic information about the technology and its anticipated effects on the planet will be shown.
109Every decision the player makes in the game has both long and short term ramifications on their progress.  The state of their progress is presented in the form of statistics on money, materials, power, space, and pollution, which are displayed to the player throughout.  When an object is first built, the player loses the associated with constructing that building, which are consider one-time costs. Then for subsequent turns afterwards, the various stats will be updated based on the on-going fees and benefits of the buildings that have been made. For instance, building a new mine will increase pollution every turn, but will also increase the amount of available materials. Thus the player must learn to juggle all these metrics simultaneously. Money will have to be used prudently and all facets of technology will have to be developed in parallel in order for the planet to prosper. Improving transportation might require better power sources, which in turn might demand more materials, and so on. Therefore the game conveys the primary theme of all aspects of technology in society being interconnected and mutually dependent. These various means of giving the player feedback on their choices achieves the learning objective and conveys the intended messages about technology to the user.
111There are many means of giving the player feedback as they play. The first of which are the buildings after they have been built. In addition to affecting a player's resources, the buildings, if clicked on, will display their stats and a brief description about the technology. Moreover, aliens on the map can be clicked on and they will voice their opinions about the buildings around them (both positive and negative). Perhaps most importantly, at the end of each turn, the player's state in the game is analyzed using a number of heuristics and events will occur in response. Furthermore, there are analyzers for the player's stats which provide suggestions about what buildings to add or delete in order to improve that respective resource. These events range from rewarding the player with more resources if they are doing well and making good choices to simulating their mines collapsing or a nuclear meltdown if they build too many nuclear power plants.\\ \\
112The actual backend design of the game separates the game logic from the graphical elements. The program design is best conveyed in the domain diagram, appended to the end of this document. This division allows for the greatest abstraction and reduces the need for these two separate game aspects from needing to interact heavily. Every graphical element subclasses from the Window class and are designed based on the intuitive elements of the game screen. Each subclass specifies the way it handles events and has its own set of responses. The game logic uses basic data-oriented classes to keep track of the player's progress. Moreover, analyzers are used for both the state and resource (spec) analysis, in order to offer dynamic suggestions and feedback through the game.
116Testing was done at all levels of development. Testing was implemented in three different ways: regression tests of the game logic, black box tests of the front end and user tests of the gameplay. These tests in tandem created a comprehensive means of evaluating, and ultimately improving, our game. Detailed descriptions as well as the results of each individual test can be found in the Test Report.
117\subsection*{Regression Tests}
118Regression tests were white box tests that tested the various components of the game logic. They were implemented as Python scripts using the \verb+unittest+ library, allowing us to automate and streamline the testing process. In particular, these scripts tested that calculations were correctly made in the classes that retain the player's game data. Among the things tested were whether a player's resources updated correctly, whether heuristics for events were triggered appropriately and if suggestions made to the player based on their progress were appropriate and as expected. Testing these aspects of the game was essential because it ensured the accuracy of the game logic and that a player's progress was correctly updated and evaluated with each decision they make. Any errors in such would have interfered with both the enjoyment of the game as well as undermine the learning objective. Moreover, it was critical that these be regression tests so that as new features were added, we would be able to quickly and rigorously gauge whether the game logic remained correct. Ensuring that all of these tests worked at every stage of develop was thus one means of assessing the correctness of our solution.
119\subsection*{Black Box Tests}
120Black box testing of the interface simulated the interaction a player would have with our game. These tests were used throughout to identify bugs in the GUI, as well as assess the game's performance. We performed both stress and endurance tests for the game's graphical elements. The stress tests attempted to expose bugs or mistakes in the design by using the game elements and windows against their intended purpose. Thus these tests looked for proper error handling for certain types of user input, as well as the general functionality of all the buttons and window elements. Endurance tests ensured that the program could withstand extending periods of use, by using the game for 10 to 20 minutes at a time. Lastly, we assessed the performance of the program by using Python's \verb+cProfile+ library to determine what parts of the program were inefficient and accounting for a lot of the processing time. This process enabled us to greatly reduce the running time of the program by optimizing the amount of rendering, image resizing, as well as internal window updating. Passing and performing well was deemed essential to verifying the quality of the game produced.
121\subsection*{User Tests}
122User tests were conducted on three occasions. The first was done on an initial user interface design that reflected the intended model of our game. The second and third instances were done using the Alpha release and a pre-Beta release. The purpose of these tests were to gather feedback on how intuitive the design was, how the clear the goals were and solicit any other feedback an outside user would offer for the game. The UI testing was conducted with middle school students and thus gave us an indication of what was clear and unclear about the layout of the window and the general gameplay. The subsequent tests were effective at determining what was confusing or vague in the overall game structure, as well as identifying areas for improvement, along with bugs. While we did not have the opportunity to test the final result with students and thus accurately assess whether we accommodated the feedback suggested throughout, all the feedback was indeed evaluated and handled in a manner intended to satisfy the most number of users. The layout and gameplay underwent many revisions as a result of this feedback.
123\subsection*{Testing Methodology}
124Regression tests were performed on a regular basis with each new feature. These were run and inspected before the feature was committed as final. Black box testing was done both informally before any change commit as well as formally each week. The formal instances were done to assess that week's work and to provide a rigorous assessment of the game's current progress. As stated, user testing was done three times, each at every different stages of development. This timing was effective in providing useful feedback for each major milestone in the project.\\ \\
125As bugs were identified, a cursory inspection of the relevant code was performed. If an immediate fix was visible, the fix was made and then committed. If not, a ticket was filed and an entry was added in the Test Report, documenting the context of the bug. As bugs were addressed, both the ticket was updated as well as the entry in the Test Report.
127There were many lessons that we learned throughout this process, ranging from user interfaces, to large software design to facts about technology. Regarding user interfaces, we learned about the importance of user testing. It is easy to become too familiar with your own work and sample users provide an abundance of insightful observations that would be otherwise overlooked. Moreover, user interfaces should aim for simplicity and elegance. Realizing the importance of these qualities significantly impacted the design of our layout and was some of the invaluable feedback received from users.\\ \\
128One of the most consequential lessons learned through the process was about extensible program design. After designing our alpha prototype, we nearly entirely scrapped out framework and started over. This opportunity allowed us to fully assess and critique what we had been using and design a new framework that would resolve and address those issues. In particular, our previous model was not conducive to adding new features. Each added element need to be properly linked and integrated nearly all other components. By better separating game logic and graphical elements, as well as using design principles about inheritance, we created a much more effective and efficient architecture that once in place supported many additions and modifications to the actual gameplay.\\ \\
129Lastly, we ourselves learned much about technology. In particular, certain technologies logically depend on others while some are independent. Moreover, we had to research into the different effects technology actually has on aspects of communities and the environment. Upon doing so, in addition to learning about it directly, we also learned that many players are quick to understand this in the context of a game, but perhaps might not think about it as much in the real world. Thus by instill this information and our message in the realm of a computer game, we provide an effective means of reaching the player and conveying the learning objective.\\ \\
130As described during their enumeration, the final game achieves a vast majority of the established requirements and al the deliverables have been produced in some form. We are very pleased with the outcome of our project. To reiterate, one area that would benefit from further improvement are making the art style more homogenous and perhaps even more professional quality. This could be easily achieved by designing all artwork from scratch using a single graphics editor, and sufficient artistic talent. Another area would be further optimizing the relative values of the building specs (perhaps through a genetic algorithm) for the most realistic and simultaneously fair gameplay. While we've ensured that the given values do permit the game to be won, we have not developed a sufficient metric for determining the feasibility of the current values used and thus we feel these offer areas of improvement. Lastly, while we did our best to accommodate all user feedback and make the game as intuitive as possible, because we were unable to conduct a final user test, we cannot say with certainty if our final result indeed is intuitive and easy to play, as was promised. However, our framework is designed to be readily adaptable should certain aspects be unclear and thus we feel we have produced not only the best game given the extent of feedback we've received, but also a highly flexible game that could be further improved, given more information.
133\bibitem{vcp} \emph{Virtual City Playground}. [Download]. United States: G5 Entertainment, 2011.
134\bibitem{mc} \emph{My Country}. [Download]. United States: Game Insight, LLC, 2011.
135\bibitem{c2} \emph{Civilization 2}. [Download]. United States: MicroProse, 1996.
136\bibitem{vd} \emph{Village Defense}. [Download]. Claremont, CA: Harvey Mudd College Games Network, 2010.
137\bibitem{milken} J Schacter. \emph{The Impact of Education Technology on Student Achievement: What the Most Current Research Has to Say}. Santa Monica, CA: Milken Exchange on Education Technology, 1999.
138\bibitem{dept} B. Means, S. Singh. ``Technology Rationale." Internet: EdReformStudies/EdTech/techrationale.html. [Feb. 3, 2012].
143   \includegraphics[width=1.4\textwidth, angle=-90]{rsc/domain_diagram.pdf}
Note: See TracBrowser for help on using the browser.