3 Copyright (C) 2003 Ushodaya Enterprised Limited
4 Author: Dan Dennedy <dan@dennedy.org>
5 Last Revision: 2004-03-20
8 NOTE: THIS DOCUMENT REQUIRES REVISION TO NEW, EXPECTED BEHAVIOR FROM MIRACLE.
10 Tests are divided into 9 sections:
14 3. Server Configuration
16 5. Multi-unit Playback
20 9. Server Side Queuing
22 Each section contains many tests which I've divided into a minimum of two lines:
24 n.m action to carry out
27 Further lines may appear to show the actual results when they deviate from what
28 I expected or if there are special cases to consider.
30 Sequential tests are indicated as:
32 n.m.o action to carry out
35 It is suggested that you run top during the testing and note cpu hikes
36 or any excessive memory usage related to an operation.
42 The tests following are by no means exhaustive, but they should cover typical
43 use cases - creativity is encouraged with more cases being added where necessary.
44 This document should also be maintained to dictate actual state, especially with
45 regard to a final release. Unit test cases are encouraged, but are excluded from
48 It is important to carry out the full test cycle when preparing a final release.
49 In this situation, please resist the temptation to bug fix a given test case and
50 resume the tests from that point onward - it is better to repeat from the
51 beginning (but you can of course employ common sense in this situation).
53 Before starting the final tests, please delete/backup your current
54 /etc/dv139d.conf file. This (more or less) ensures that tests are carried out
61 Run these from the top level project directory
63 1.1.0 Start miracle in interactive mode: src/miracle/miracle -test
64 --> miracle starts interactively and reports:
65 (5) Starting server on 5250.
66 (5) miracle version 0.0.1 listening on port 5250
68 1.1.1 Stop the server by pressing Ctrl-C
69 --> miracle returns the following and returns control to the console:
70 (5) miracle version 0.0.1 server terminated.
72 1.2.2 Start miracle as a daemon: src/miracle/miracle
73 --> control returns to the console
75 1.2.3 Verify miracle is running: ps ax
76 --> several miracle processes are running
78 1.2.4 Verify successful miracle startup using syslog: sudo tail /var/log/syslog
79 --> miracle: miracle version 0.0.1 listening on port 5250
81 1.2.5 Verify connectivity on port 5250: telnet localhost 5250
84 1.2.6 Test clean disconnect: BYE
85 --> Connection closed by foreign host.
87 1.2.7 Stop the daemon: killall miracle
90 1.2.8 Verify a clean server shutdown: sudo tail /var/log/syslog
91 --> miracle: shutdown complete.
93 1.3.0 Start miracle on a different port: src/miracle/miracle -port 5260
95 1.3.1 Verify successful miracle startup using syslog: sudo tail /var/log/syslog
96 --> miracle: miracle version 0.0.1 listening on port 5260
98 1.3.2 Verify connectivity on port 5260: telnet localhost 5260
101 1.3.3 Test clean disconnect: BYE
102 --> Connection closed by foreign host.
104 1.3.4 Stop the daemon: killall miracle
111 Start the miracle server and connect to it with telnet or a protocol-
112 level client (albino).
114 2.1 List the AV/C devices on the bus: NLS
116 --> one or more lines listing devices with GUID in second column
118 2.2 Add a device as a miracle unit: UADD {sdl, bluefish}
122 2.3 List the units: ULS
124 --> U0 ?? {sdl, bluefish} 1
125 --> It is important that the last column is '1' to indicate it is online.
127 2.4 List the units: ULS
129 --> U0 ?? {sdl, bluefish} 1
131 2.5 Attempt unit commands for a unit that does not exist: LOAD U6 foo
132 --> 403 Unit not found
134 2.6 Attempt unit commands without specifying a unit: PLAY
135 --> 402 Argument missing
137 2.7 Attempt unit commands for a unit: PLAY U0
140 2.8.0 Load a clip into an unit: LOAD U0 test.dv
143 2.7.1 Verify the status of the unit: USTA U0
145 --> 0 online "test.dv" 0 1000 25.00 0 ...
146 --> only the first 3 columns are relevant in this test
149 3. Server Configuration
150 -----------------------
152 Start miracle if not already started from a previous test.
154 3.1 Get the hard-coded default root property value: GET root
158 3.2 List the files and subdirectories at the root: CLS /
163 3.3 Change the server root to a place where clips are stored: e.g.,
167 3.4 Get the new value of the root property value: GET root
170 --> Notice that if you did not use a trailing slash in step 2.3, one is
171 added for you and appears in this step. This is normal and correct.
173 3.5 List the files and subdirectories at the root: CLS /
175 --> zero or more lines listing subdirectories followed by files.
177 3.6 Try to set a property that does not exist: SET foo=bar
178 --> 405 Argument value out of range
180 3.7 Try to set no property or value: SET
181 --> 402 Argument missing
183 3.8 Attempt a bogus command: FOO
184 --> 400 Unknown command
186 XXX 3.9 Attempt the incorrect case for a command: get root
187 XXX --> 400 Unknown command
189 3.10 Attempt case insensitivity of property key: GET Root
197 Start miracle or restart if already started.
199 Set the server root property if desired.
201 4.1.0 Load a clip into the unit: LOAD U0 test.dv
204 4.1.1 Check the unit status: USTA U0
206 --> 0 stopped "test.dv" 0 1000 25.00 0 ...
207 --> Only the first 3 columns are relevant in this test.
208 --> The remaining columns are only relevant to the tester.
210 4.2.0 Play the clip: PLAY U0
212 --> Verify audio and video output
214 4.2.1 Check the unit status: USTA U0
216 --> 0 playing "test.dv" 1739 1000 25.00 0 ...
217 --> Only the first 3 columns are relevant in this test.
218 --> The remaining columns are only relevant to the tester.
220 4.3.0 Pause playback: PAUSE U0
222 --> Verify video continues, but audio is muted.
224 4.3.1 Check the unit status: USTA U0
226 --> 0 paused "test.dv" 1739 0 25.00 0 ...
227 --> The fifth column --------^ should be 0; it indicates speed.
229 4.3.2 Stop playback: STOP U0
231 --> The analog video output stops
233 4.3.3 Pause playback: PAUSE U0
235 --> Analog video starts again, but it is held on the same frame
238 4.3.4 Stop playback: STOP U0
240 --> The analog video signal ceases.
242 4.3.5 Rewind the unit: REW U0
245 4.3.6 Play the unit: PLAY U0
247 --> Analog audio and video are produced from the beginning of the file.
249 4.4 Stop the server during playback and ensure clean shutdown.
252 5. Multi-unit Playback
253 -----------------------
255 Start miracle or restart if already started.
256 Add *2* online units.
257 Set the server root property if desired.
259 5.1.0 Load a clip into one unit: LOAD U0 test.dv
262 5.1.1 Load a clip into the other unit: LOAD U1 test.dv
265 5.1.2 Start playing one unit: PLAY U0
267 --> Verify audio and video output
269 5.1.3 Start playing the other unit: PLAY U1
271 --> Verify audio and video output of both units
273 5.2 Verify independence of units by pausing one of them: PAUSE U0
275 --> Verify video continues, but audio is muted on the first unit only.
277 5.3 Stop the server during multi-unit playback and ensure clean shutdown.
283 Start miracle or restart if already started.
284 Add *2* online units.
285 Set the server root property if desired.
289 6.1.0 Load a clip: LOAD U0 test.dv
292 6.1.1 Start playback by pausing on the first frame: PAUSE U0
294 --> analog video starts, but audio is muted.
296 6.1.2 Play fast forward: FF U0
298 --> verify video is playing fast in the forward direction.
300 6.1.3 Get unit status: USTA U0
302 --> 0 playing "test.dv" 219 2000 25.00 0 ...
303 --> The important column is --^, indicates speed
305 6.1.4 Play fast reverse: REW U0
307 --> verify analog video output is fast in the reverse direction.
309 6.1.5 Get unit status: USTA U0
311 --> 0 playing "test.dv" 4621 -2000 25.00 0 ...
312 --> The important column is ---^, negative mean reverse
314 6.1.6 Play slow forward: PLAY U0 500
316 --> Verify the analog video output is slow in the forward direction.
318 6.1.7 Play reverse normal speed: PLAY U0 -1000
320 --> Verify the analog video output is at a normal speed in the reverse direction.
321 --> Audio output is reverse, but not the field order of video
323 Loading while playing:
325 6.2.0 Stop the unit (might be playing): STOP U0
328 6.2.1 Rewing the unit: REW U0
331 6.2.2 Start playing: PLAY U0
333 --> verify analog audio and video output
335 6.2.3 Load another clip: LOAD test002.dv
337 --> playback seamlessly switches to the new clip
338 --> verify the analog appearance of the video makes a clean switch
340 6.2.4 Load another clip, this time with in and out points:
341 LOAD test.dv 100 500 (whatever works for your test footage)
343 --> verify the analog appearance of the video makes a clean switch
345 6.2.5 Get unit status: USTA U0
347 --> 0 playing "test.dv" 403 1000 25.00 100 ...
348 --> verify position -----^ is beyond --^ in point, last column is the out
349 point specified in the previous step.
353 6.3.0 Load a clip: LOAD U0 test.dv
356 6.3.1 Pause the playback unit: PAUSE U0
359 6.3.2 Set the in point: SIN U0 100
361 --> verify the frame displayed in analog video out changes
363 6.3.4 Get the unit status: USTA U0
365 --> 0 paused "test.dv" 100 0 25.00 100 ...
366 --> verify position ---^ and in ---^
368 6.3.5 Change the mode of the unit to not restrict playback to the edit
369 region: USET U0 points=ignore
372 6.3.6 Jump to a frame before the in frame: GOTO U0 50
375 6.3.7 Get the unit status: USTA U0
377 --> 0 paused "test.dv" 50 0 25.00 100 ...
378 --> position ----------^ preceeds -^ (in)
380 6.3.8 Set the unit mode to restrict playback to the edit region: USET U0 points=use
382 --> verify frame on analog video output changes
384 6.3.9 Get the unit status: USTA U0
386 --> 0 paused "test.dv" 100 0 25.00 100 ...
387 --> verify position ---^ and in ---^
389 6.3.10 Clear the in point: SIN U0 -1
392 6.3.11 Get the unit status: USTA U0
394 --> 0 paused "test.dv" 100 0 25.00 0 ...
395 --> verify the in point is reset --^
397 The above sequence should be repeated in a similar manner for the out point
398 using the SOUT command.
402 6.4.0 Load a clip into the first unit: LOAD U0 test.dv
405 6.4.1 Load a clip into the second unit: LOAD U1 test002.dv
408 6.4.2 Start playing the first unit: PLAY U0
410 --> verify audio and video analog output
412 6.4.3 Set an in point on the clip in the second unit: SIN U1 100
415 6.4.4 Play the second unit: PLAY U1
417 --> note the beginning footage
419 6.4.5 Transfer the clip from the second to the first unit: XFER U1 U0
421 --> verify a clean switch on the analog audio and video output of the first unit.
422 --> upon transfer it should play the same footage previewed in step 6.4.4
424 6.4.5 Get the first unit's status: USTA U0
426 --> 0 playing "test002.dv" 963 1000 29.97 100 2502
427 --> note the in point set from U1 ---------^
430 7. Unit Configuration
431 ---------------------
433 Start miracle or restart if already started.
435 Set the server root property if desired.
437 7.1.0 Load a short clip: LOAD U0 test.dv
440 7.1.1 Play a clip: PLAY U0
442 --> Wait until it gets to the end, and it should pause on the last frame.
444 7.1.2 Make the clip start looping: USET U0 eof=loop
446 --> verify the clip starts playing from the beginning and loops
448 7.2.0 Set the in point: SIN U0 10
450 --> playback pauses at in point (verify with USTA U0)
452 7.2.1 Set the out point: SOUT U0 200
454 --> playback pauses at in point (verify with USTA U0)
456 7.2.2 Start playing again: PLAY U0
458 --> verify playback loops between in and out points
460 7.3 Tell the unit to ignore the edit points: USET U0 points=ignore
462 --> verify playback loops over entire video file
464 7.4 Get the current value of the points property: UGET U0 points
469 9. Server Side Queuing
470 ----------------------
472 Only one unit is used for these test cases, and
473 users are encouraged to test with multiple units online. It is assumed that a
474 number of dv files are available for use in the servers ROOT directory - this
475 document assumes that they are named test001.dv and up.
477 9.1.0 Start miracle in interactive mode and add a unit (all tests will assume U0)
478 --> server started with unit 0 available
480 9.1.1 Obtain a miracle shell (via telnet or albino).
481 --> 100 VTR (if reported by the client)
483 9.1.2 Load a clip with LOAD U0 test001.dv and PAUSE U0
486 9.1.3 List the clips with LIST U0
489 --> 0 "test001.dv" 0 6999 7000 7000 25.00
490 --> The 1 on the second line denotes the number of times the list has been changed
491 via user commands (known as the 'generation' number).
492 --> The third line and beyond reports the clip index (from 0 to n), file loaded, in point,
493 out point, real size of the file and the calculated size (out - in + 1 ).
495 9.1.4 Check the unit status with USTA U0
497 --> 0 paused "test001.dv" 0 0 25.00 0 6999 7000 "test001.dv" 0 0 6999 7000 1 1 0
498 --> The last two fields indicate the generation number and current clip resp.
500 9.1.5 Append a clip with APND U0 test002.dv followed by LIST U0
503 --> 0 "test001.dv" 0 6999 7000 7000
504 --> 1 "test002.dv" 0 6999 7000 7000
505 --> Check that USTA U0 reports a generation of 2 and current clip of 0
507 9.1.6 Move clip 1 to clip 0 with MOVE U0 1 0 followed by LIST U0
510 --> 0 "test002.dv" 0 6999 7000 7000
511 --> 1 "test001.dv" 0 6999 7000 7000
512 --> Check that USTA U0 reports a generation of 3 and current clip of 1
514 9.1.7 Move clip 0 to clip 1 with MOVE U0 0 1 followed by LIST U0
517 --> 0 "test001.dv" 0 6999 7000 7000
518 --> 1 "test002.dv" 0 6999 7000 7000
519 --> Check that USTA U0 reports a generation of 4 and current clip of 0
520 --> Note that the order in which you run 9.1.6 and 9.1.7 shouldn't matter as the
521 result will be identical
523 9.1.8 Change the position to the next clip with GOTO U0 0 +1
525 --> Check that USTA U0 reports a generation of 4 and current clip of 1
527 9.1.9 Remove all but the playing clip with CLEAN U0 followed by LIST U0
530 --> 0 "test002.dv" 0 6999 7000 7000
531 --> Check that USTA U0 reports a generation of 5 and current clip of 0
533 9.1.10 Insert test001.dv back into the list using INSERT U0 test001.dv and run LIST U0
536 --> 0 "test001.dv" 0 6999 7000 7000
537 --> 1 "test002.dv" 0 6999 7000 7000
538 --> Check that USTA U0 reports a generation of 6 and current clip of 1
540 9.1.11 Insert test003.dv at position 2 using INSERT U0 test001.dv 3 and run LIST U0
543 --> 0 "test001.dv" 0 6999 7000 7000
544 --> 1 "test002.dv" 0 6999 7000 7000
545 --> 2 "test003.dv" 0 6999 7000 7000
546 --> Check that USTA U0 reports a generation of 7 and current clip of 1
548 9.1.12 Change the in point of the current clip using SIN U0 5000 and run LIST U0
551 --> 0 "test001.dv" 0 6999 7000 7000
552 --> 1 "test002.dv" 5000 6999 7000 2000
553 --> 2 "test003.dv" 0 6999 7000 7000
554 --> Check that USTA U0 reports a generation of 8 and current clip of 1
556 9.1.13 Change the out point of the following clip using SOUT U0 5000 +1 and run LIST U0
559 --> 0 "test001.dv" 0 6999 7000 7000
560 --> 1 "test002.dv" 5000 6999 7000 2000
561 --> 2 "test003.dv" 0 5000 7000 5001
562 --> Check that USTA U0 reports a generation of 9 and current clip of 2
564 9.1.14 Change the in point of the current clip to 1000 using SIN U0 1000 and run LIST U0
567 --> 0 "test001.dv" 0 6999 7000 7000
568 --> 1 "test002.dv" 5000 6999 7000 2000
569 --> 2 "test003.dv" 1000 5000 7000 4001
570 --> Check that USTA U0 reports a generation of 10 and current clip of 2
572 9.1.15 Ignore the in/out points by running USET U0 points=ignore and run LIST U0
575 --> 0 "test001.dv" 0 6999 7000 7000
576 --> 1 "test002.dv" 5000 6999 7000 7000
577 --> 2 "test003.dv" 1000 5000 7000 7000
578 --> Check that USTA U0 reports a generation of 11 and current clip of 2
580 9.1.16 Turn the in/out points on again by running USET U0 points=use and run LIST U0
583 --> 0 "test001.dv" 0 6999 7000 7000
584 --> 1 "test002.dv" 5000 6999 7000 2000
585 --> 2 "test003.dv" 1000 5000 7000 4001
586 --> Check that USTA U0 reports a generation of 12 and current clip of 2
588 9.1.17 Remove the current clip using REMOVE U0 and run LIST U0
591 --> 0 "test001.dv" 0 6999 7000 7000
592 --> 1 "test002.dv" 5000 6999 7000 2000
593 --> Check that USTA U0 reports a generation of 13 and current clip of 0
595 9.1.17 Remove the next clip using REMOVE U0 +1 and run LIST U0
598 --> 0 "test001.dv" 0 6999 7000 7000
599 --> Check that USTA U0 reports a generation of 14 and current clip of 0