This document defines the plan for functional testing of POSIX Clocks
and Timers.
First a list of the features that will and will not be tested is given.
Then, an attempt is made to break down the features into possible testing
scenarios. Not all of these may be implemented. The assertions.xml file
will have the additional detail on which were chosen for implementation.
Features To Be Tested
=====================
The following features will be tested:
clocks - Testing that clocks can accurately keep time in certain scenarios.
sleep - Testing that processes can go to sleep for specified periods of time.
timers - The following types of timers will be tested.
absolute timer, no repeat think: alarm clock
relative timer, no repeat think: stopwatch
absolute timer, repeats think: grandfather clock
relative timer, repeats think: repeating stopwatch
Also, timer overruns will be counted.
Features Not To Be Tested
=========================
API testing of the clocks/timers functions is considered to be done by
the conformance section.
Load and other stress testing will be done in the stress section.
A few error handling tests will be created, but the majority will not for
this initial release. (Over time, these may be added from regression
tests, or regression tests may suggest additional error handling tests to
create.)
Approach
========
In order to generate the test cases listed below, the following approaches
were taken:
- If it was not covered in the conformance area fully, determined the
correct behavior of the features listed above and generated test scenarios
to verify this behavior happened.
- Determined certain critical values that may cause failures to appear and
generated test scenarios setting clocks or timers to these critical
values.
- Determined race conditions that could appear or time dependencies for
things happening and generated test cases to expose these issues.
Test Case Scenarios
===================
Some potential test case scenarios are listed after each functional
area.
Clocks
------
- Set clock time to: 0, a large number, all the dates that were
critical for Y2K testing, set to time before daylight savings (and then
after daylight savings). ==> This is probably multiple test cases.
- Call clock_gettime() mulitple times and ensure the time is always
increasing (call also at nsec increments to ensure both nsec and sec
are always increasing).
- Ensure we can tell time within acceptable jitter (over some period of time).
- Test for a few different clocks.
- Test after setting the time.
- Send a signal while setting the time.
- Kill the process (example of previous with SIGKILL) when setting the time.
- Have a timer expire when setting the time.
- Set the time from two different processes.
Sleep
-----
- Mimic the non-repeating timers tests as they apply to sleep.
Timers
------
Non-repeating (relative and absolute)
- Set timer expiration to: 0, a large number, all the dates that were
critical for Y2K testing, set to time before daylight savings (and then
after daylight savings). ==> This is probably multiple test cases. Some
are absolute timers only.
- Have the clock change underneath a timer and ensure it expires
correctly.
- [Relative only] Set up multiple timers to expire after the same duration
and ensure they expire in the order set up. (May need to nanosleep
for CLOCKRES nanoseconds in between set ups to get correct results. Also,
even if this fails, not necessarily a "bug" since many factors influence
expiration time.)
- Test out all the various events that can be sent for a timer expiration.
- Set two timers to expire at the same time, different events, in the
same process.
- Set up two timers to expire for the same time in two different processes.
- Set up one repeating timer to expire after X time and one non-repeating
to expire after 2*X time.
Repeating (relative and absolute)
- Perform all non-repeating timer tests on repeating timers.
- Ensure that these are still going off at the same time, over time.
Overruns
- Have multiple timers in a process blocked and ensure overrun count
for each is correct.
- Have a repeating timer blocked for a period of time and ensure overrun
count is correct.
- Ensure overrun count and timer_gettime() return correct values when
SIGEV_NONE is used in ev.sigev_notify.
- Try to call timer_getoverrun() as an overrun is happening.
Contributors: julie.n.fleischer REMOVE-THIS AT intel DOT com
geoffrey.r.gustafson REMOVE-THIS AT intel DOT com
rolla.n.selbak REMOVE-THIS AT intel DOT com
majid.awad REMOVE-THIS AT intel DOT com
salwan.searty REMOVE-THIS AT intel DOT com
george REMOVE-THIS AT mvista DOT com