DELTA 145 0 350 SVN 5 Mwh%V\documentclass[11pt,letterpaper]{article} \usepackage[margin=1in]{geometry} \renewcommand{\labelenumi}{\arabic{section}.\arabic{enumi}.} % LastModified: $Id$ \author{Brett McLarnon} \title{ \textbf{Test Plan} \\ Calendaring Import/Export ModuleThe general testing strategy for the Calendaring Import and Export modules is to convert a known calendar into another format. Existing tools (either provided by other pieces of Calendaring or by external libraries) are then used to iterate over both versions of the calendar to compare the events and todo items in each. The differences must then be analyzed to determine whether the degree of lossiness is acceptable given the particular conversion: some formats are inherently less expressive than others, and so a lossless conversion is not always possible. It is up to the specific test to determine whether any of the discrepancies are unjustified; if they are, the test should fail. Otherwise, the conversion is considered a success. In addition, calendar imports are allowed to be lossy as long as they identify the places at which information loss occurred. This information must be checked to confirm that all instances of data loss were correctly flagged and that each instance is unavoidable---or at least acceptable. Each test case should be written as its own class implementing a generic Test interface, which would require \texttt{setup()}, \texttt{run()}, and \texttt{teardown()} methods, as well as display methods to produce an informative error message when the test failed (i.e., the name and purpose of the failed test). With such a model, it should be easy to write a framework that just looks runs each test and logs the results. Or, with minimal effort, it should be possible to use these test case classes with existing xUnit compliant automation frameworks. The ultimate choice of which framework to use should depend on the implementation language of the Calendaring component; the test cases are described using pseudocode, simplifying the process of porting them to desired framework, be it JUnit, CUnit, NUnit (for .NET), or some other similar tool. \section{Stubbed Components} Testing these modules requires that the Calendar, EventSeries, TodoList, and TodoItem classes be implemented sufficiently to allow construction and data access. In addition, the Pattern module must be fully specified so that proper interaction with it can be confirmed. Since the Calendar, EventSeries, TodoList, and TodoItem classes provide few methods other than their constructors and accessors, there is little benefit to developing separate stubbed versions. Instead, the mainline versions can be branched; the methods necessary for testing import and export (the constructors and accessors) are very unlikely to contain bugs, and so it would be a waste of time to rewrite them. The additional, more complicated methods these classes provide (such as conversion of Calendars to FixedCalendars) need not be implemented. Thus, little to no time need be spent on constructing stubbed components, assuming that basics of these classes have been implemented. Note that the Pattern component must have all of the subclasses defined in \texttt{/tags/proj3/architecture/pattern/DESIGN}; import and export has its greatest dependence on Patterns, and these classes can not be effectively stubbed (any stub would have to implement most if not all of the functionality of the real class). \section{Acceptance Test Cases} The following are component requirements developed during project 3 for the Calendaring Import/Export module\footnote{These requirements were taken verbatim from the CalendarImporter and CalendarExporter sections of \texttt{/tags/proj3/architecture/COMPONENT\_SPECIFICATION}.}: \begin{itemize} \item Import calendars from standard formats, creating the necessary Calendar and TodoList objects. \item Provide easy addition of new import formats. \item Export Calendars and TodoLists into standard formats. \item Provide easy addition of new export formats. \end{itemize} The second and fourth requirement are subjective maintainability requirements rather than testable functional requirements; they are best confirmed using design review rather than automated testing. The first and third requirements therefore become the basis for our acceptance test cases. From these two requirements, we obtain the following tests to confirm that the component performs all the specified functions and handles obvious errors: \begin{enumerate} \item CalendarImporter: Valid Import \begin{itemize} \item Purpose: Valid calendars in each of the supported external formats should be able to be imported into a valid Calendar and TodoList pair. These ``valid'' calendars will not contain any expressions that are not representable using Calendars and TodoLists. \item Set-Up: Have existing valid calendars for each of the given supported import formats. Furthermore, we need to have parsers for those formats---Open Source parsers exist for iCalendar for languages including PHP, Python, and Ruby. \item Operation: For each import format, create a new CalendarImporter object and have it import the correct file. \item Results: For each event in the resulting Calendar object, make sure that both the EventSeries and the event as reported by an existing parser have the same attribute values (name, description, lifespan, start date and time, end date and time, priority, and busy time indication) for all iterations (if the recurrence isn't infinite). If the recurrence is infinite, then make sure that the events are identical for at least 100 iterations. Similarly, we check to confirm that attributes are identical for all todo items. If all events and todo items are identical, then the import was successful. \end{itemize} \item CalendarImporter: Invalid File Detection \begin{itemize} \item Purpose: An exception should be thrown when an invalid calendar/todo list is imported from each supported format. \item Set-Up: Have (perhaps several) invalid calendars for each of the import formats. These files should range from very broken (invalid headers) to slightly broken (omitted or mangled event termination). \item Operation: Construct a CalendarImporter for each import format, and then try to import the broken file (surrounded by a try-catch block). \item Results: If an exception is thrown, the test passes. Otherwise, it fails. \end{itemize} \item CalendarExporter: Valid Export \begin{itemize} \item Purpose: A Calendar and TodoList pair exported to each supported format should conform to the specification for that format. \item Set-Up: Construct Calendar and TodoList pairs with varying numbers of EventSeries and TodoItems, respectively. \item Operation: Export each Calendar and TodoList pair to each of the supported formats, then use the parser for that language to parse it (in its strictest mode). \item Results: If the parsers do not throw any errors/exceptions, then the output file should be valid and the test passes. The files should also be checked to be non-empty (i.e., the condition doesn't succeed trivially). \end{itemize} \item CalendarExporter: Invalid Object Detection \begin{itemize} \item Purpose: An error should be produced when an invalid Calendar object or TodoList object is exported to each supported format. \item Set-Up: Construct corrupted Calendar and TodoList objects. \item Operation: For each export format, construct a CalendarExporter and try an export, catching any exceptions. \item Results: If exceptions are thrown, the test passes. If the exporter for some format doesn't detect the invalid object, then no exception will be thrown and the test should fail. \end{itemize} \end{enumerate} \section{Functionality Test Cases} From the component specification\footnote{See the specification portions of \texttt{/tags/proj3/architecture/import\_export/DESIGN}.} we derive the following additional test cases: \begin{enumerate} \item CalendarImporter: Bad Argument Detection \begin{itemize} \item Purpose: The \texttt{import()} method should detect if it is given an invalid input stream. \item Set-Up: \emph{None.} \item Operation: Construct a CalendarImporter and call \texttt{import()} with an input stream that will fail on a read (it could be at the end of the file, it could refer to a missing file, etc.). \item Results: If an exception is thrown, the test passes. If the method fails silently or breaks, the test should fail. \end{itemize} \newpage \item CalendarImporter: No Input Handling \begin{itemize} \item Purpose: The methods \texttt{getCalendar()} and \texttt{getTodoList()} should return empty Calendars and TodoLists, respectively, when no data has been imported. Similarly, the methods \texttt{getUnreadableEvents()} and \texttt{getUnreadableTodoItems()} should return the appropriate empty objects. \item Set-Up: Valid empty calendar files for each format. \item Operation: For each supported format, construct a CalendarImporter, then obtain the results of the methods \texttt{getCalendar()}, \texttt{getTodoList()}, \texttt{getUnreadableEvents()}, and \texttt{getUnreadableTodoItems()}. Similarly, try importing empty files for each format. \item Results: If any of the obtained objects are not the equivalent of an empty calendar, todo list, or collection of events or todo items, the test fails. \end{itemize} \item CalendarImporter: Multiple Input Support \begin{itemize} \item Purpose: Multiple calls to \texttt{import()} should combine the calendars. \item Set-Up: Have multiple valid calendars for each format. \item Operation: Construct a CalendarImporter and import all the files. \item Results: Use the method outlined in \emph{CalendarImporter: Valid Import} to make sure that each EventSeries corresponds to an event (and each TodoItem to a todo item) in one of the input files. If all events and todo items match, the test passes. \end{itemize} \item CalendarImporter: Unreadable Object Identification \begin{itemize} \item Purpose: Malformed events and todo items should be flagged when a calendar is imported. These problematic events and todo items must be known. \item Set-Up: Have calender files containing some unparseable/unrepresentable events and todo items. \item Operation: Import each each file, then obtain the results of \texttt{getUnreadableEvents()} and \texttt{getUnreadableTodoItems()}. \item Results: Make sure that the known unidentifiable events and todo items are correctly identified and returned by \texttt{getUnreadableEvents()} and \texttt{getUndreadableTodoItems()}. If they are not all present or if additional ones are present, the test fails. \end{itemize} \item CalendarExporter: Bad Argument Detection \begin{itemize} \item Purpose: The \texttt{export()} method should detect if it is given an invalid output stream. \item Set-Up: \emph{None.} \item Operation: Construct a CalendarExporter and call \texttt{export()} with an output stream that will fail on a write (fake a full disk, etc.). \item Results: If an exception is thrown, the test passes. If the method fails silently or breaks, the test should fail. \end{itemize} \item CalendarExporter: No Input Handling \begin{itemize} \item Purpose: \texttt{export()} should produce an empty calendar if no Calendar or TodoList data has been added. \item Set-Up: Construct an empty Calendar and TodoList. \item Operation: For each supported format, construct a CalendarExporter. Obtain the results of \texttt{export()}. Similarly, add empty Calendar's and TodoLists before calling \texttt{export()}. \item Results: If any of the resulting files are not the equivalent of an empty calendar (as determined by the existing parsers), then the test fails. \end{itemize} \item CalendarExporter: Multiple Input Support \begin{itemize} \item Purpose: Multiple calls to \texttt{addCalendar()} and \texttt{addTodoList()} should combine the input. \item Set-Up: Have multiple Calendars and TodoLists. \item Operation: Construct a CalendarExporter for each of the export formats, then call \texttt{addCalendar()} and \texttt{addTodoList()} multiple times. Export the calendar. \item Results: Use the existing parsers to look at each output file. Make sure that every event and todo item in the input Calendars and TodoLists can be found in the output file (we can just compare names, since we should be able to assume that general output works). If every event and todo item can be found, then the test passes. \end{itemize} \item CalendarExporter: Export Completeness \begin{itemize} \item Purpose: The exported calendar should retain as much information as the format allows about each event (i.e., minimal lossiness). \item Set-Up: Have existing populated Calendars and TodoLists. \item Operation: Construct a CalendarExporter for each of the export formats, add the provided Calendar and TodoList, then call \texttt{export()}. \item Results: Traverse the output file using an existing parser. Make sure that each event at least has the name, description, and times of the originating event (iterate over an entire series if it is finite, or 100 iterations if it is infinite). Additional attributes may be checked if the format supports it (e.g., iCalendar supports priorities). If there is any deviation, the test fails. \end{itemize} \end{enumerate} \section{White Box Test Cases} Using the module design\footnote{See the design portions of \texttt{/tags/proj3/architecture/import\_export/DESIGN}.}, we obtain the following white box test cases: \begin{enumerate} \item ICalendarImporter: Multiple Recurrence String Support \begin{itemize} \item Purpose: Events with multiple recurrence strings and exclude dates should not gain or lose dates when converted. \item Set-Up: Have an iCalendar file with events with multiple recurrence strings. \item Operation: Import these files. \item Results: Iterate over all of the events and make sure that the dates match (i.e., no format has an event scheduled when the other does not). If such a discrepancy exists, the test fails. \end{itemize} \item ICalendarPatternImporter: Needing Multiple Patterns \begin{itemize} \item Purpose: A recurrence pattern that requires multiple Pattern objects to represent should be accurately captured. \item Set-Up: Have recurrences that require multiple Patterns to represent. \item Operation: Convert all of the recurrence strings to Pattern objects. \item Results: Iterate over the event instances (using an external parser for the iCalendar recurrence string), making sure that both representations have the same event times and dates. \end{itemize} \item ICalendarPatternImporter: Unrepresentable Recurrences \begin{itemize} \item Purpose: An UnrepresentableRecurrence exception should be thrown when an unrepresentable recurrence is imported. \item Set-Up: Recurrences currently unable to be represented by the Pattern class. \item Operation: Attempt to convert the recurrences to a Pattern. \item Results: If an UnreprestableReccurence exceptions are not thrown, fail. \end{itemize} \item ICalendarPatternImporter: Exception Dates \begin{itemize} \item Purpose: Exception information should be carried over to the resulting Pattern even when multiple Patterns are needed to represent the recurrence. \item Set-Up: Have basic recurrences with exception dates and recurrences that require multiple Patterns to represent with exception dates. \item Operation: Convert all of the recurrence strings to Pattern objects. \item Results: Iterate over all the event instances (using an external parser for the iCalendar recurrence string), making sure that both representations have the same event times and dates. \end{itemize} \item ICalendarPatternExporter: Pattern Conversion \begin{itemize} \item Purpose: iCalendar recurrence strings should be converted to equivalent Pattern objects. \item Set-Up: Recurrence strings corresponding to each Pattern type. \item Operation: Call \texttt{export()} with each of the recurrence strings. \item Results: For each recurrence string, compare against hard-coded results to make sure that the right Pattern type was generated with the right parameters. \end{itemize} \item ICalendarPatternExporter: Exception Date Transfer \begin{itemize} \item Purpose: Exception dates should be transferred from the Pattern object to the recurrence. \item Set-Up: iCalendar recurrence strings with known exception dates. \item Operation: Call \texttt{export()} with each of the recurrence strings. \item Results: For each recurrence string, make sure that the known exception dates don't have events scheduled on them by using \texttt{Pattern.getNextInterval()}. \end{itemize} \newpage \item ICalendarDateTimeConverter: Normal and Extremal Conversion \begin{itemize} \item Purpose: Conversions of ``normal'' dates (in both directions) and distant dates (far past and far future) should be consistent and stable. \item Set-Up: \emph{None.} \item Operation: Using some contemporary and some distant dates, convert them back and forth between formats. \item Results: If the resulting dates are equivalent after conversion to and from the other format, then the conversion is very likely to be correct (since the validity of the UTC representation is checked by other tests). \end{itemize} \item ICalendarDataTimeConverter: Invalid UTC Dates \begin{itemize} \item Purpose: \texttt{dateFromString()}, \texttt{timeFromString()}, and \texttt{dateTimeFromString()} should throw exceptions when the provided string is not a valid UTC time. \item Set-Up: Known invalid UTC time strings. \item Operation: Try to convert the given strings. \item Results: Catch exceptions; if an exception is caught for every conversion, the test should pass. Otherwise, fail. \end{itemize} \end{enumerate} \end{document} ENDREP id: 17.w.r161/18813 type: file pred: 17.w.r159/0 count: 8 text: 161 0 18790 18869 e01adc49ab179bbb88c47da05816dfa2 props: 145 363 66 0 6ba7dd82ad93517d00415b716ccdc1bf cpath: /trunk/testing/brett/TestPlan.tex copyroot: 159 /trunk/testing/brett/TestPlan.tex PLAIN K 12 TestPlan.tex V 20 file 17.w.r161/18813 END ENDREP id: 16.0.r161/19132 type: dir pred: 16.0.r159/315 count: 8 text: 161 19071 48 48 0c927abf4fa8c0871a84f53ca3177205 props: 145 701 54 0 2861bd3d36f04f7976bb7db1d611bdb5 cpath: /trunk/testing/brett copyroot: 0 / PLAIN K 5 brett V 19 dir 16.0.r161/19132 K 4 mike V 18 dir 18.0.r160/2999 END ENDREP id: 15.0.r161/19427 type: dir pred: 15.0.r160/3237 count: 16 text: 161 19342 72 72 5d544898b82cedb0c0fa60686f614d5c cpath: /trunk/testing copyroot: 0 / PLAIN K 12 architecture V 18 dir m.0.r132/10273 K 12 requirements V 16 dir j.0.r53/2435 K 7 testing V 19 dir 15.0.r161/19427 END ENDREP id: 3.0.r161/19716 type: dir pred: 3.0.r160/3522 count: 145 text: 161 19580 123 123 9204ce58d8705426182156ea7261ec7a cpath: /trunk copyroot: 0 / PLAIN K 8 branches V 17 dir 1.0.r143/2096 K 4 tags V 16 dir 2.0.r144/236 K 5 trunk V 18 dir 3.0.r161/19716 END ENDREP id: 0.0.r161/19980 type: dir pred: 0.0.r160/3782 count: 161 text: 161 19862 105 105 206f4ddb517c3ae5a7dc2aa38b23a64f cpath: / copyroot: 0 / 17.w.t160-1 modify true false /trunk/testing/brett/TestPlan.tex 19980 20121