Skip to content

Commit 6125b76

Browse files
philipandaphilipanda
committed
measurements.robot: Use wrapper list of tests instead of canaries
Signed-off-by: Filip Gołaś <[email protected]>
1 parent 87247d2 commit 6125b76

File tree

1 file changed

+85
-107
lines changed

1 file changed

+85
-107
lines changed

‎dasharo-performance-parallelable/measurements.robot‎

Lines changed: 85 additions & 107 deletions
Original file line numberDiff line numberDiff line change
@@ -1,114 +1,53 @@
11
*** Settings ***
2-
Library Collections
3-
Library OperatingSystem
4-
Library Process
5-
Library String
6-
Library Telnet timeout=20 seconds connection_timeout=120 seconds
7-
Library SSHLibrary timeout=90 seconds
8-
Library RequestsLibrary
9-
Library DateTime
2+
Library Collections
3+
Library DateTime
4+
Library OperatingSystem
5+
Library Process
6+
Library String
7+
Library Telnet timeout=20 seconds connection_timeout=120 seconds
8+
Library SSHLibrary timeout=90 seconds
9+
Library RequestsLibrary
1010
# TODO: maybe have a single file to include if we need to include the same
1111
# stuff in all test cases
12-
Resource ../variables.robot
13-
Resource ../keywords.robot
14-
Resource ../keys.robot
15-
Resource ../keys-and-keywords/heads-keywords.robot
16-
Resource ../lib/performance/cpu.robot
17-
Resource ../lib/sensors/sensors.robot
12+
Resource ../variables.robot
13+
Resource ../keywords.robot
14+
Resource ../keys.robot
15+
Resource ../keys-and-keywords/heads-keywords.robot
16+
Resource ../lib/performance/cpu.robot
17+
Resource ../lib/sensors/sensors.robot
18+
1819
# TODO:
1920
# - document which setup/teardown keywords to use and what are they doing
2021
# - go threough them and make sure they are doing what the name suggest (not
2122
# exactly the case right now)
23+
Suite Setup Run Keywords
24+
... Prepare Parallel Test Suite
25+
# ... Prepare Test Suite AND
26+
# ... Run Keyword If "CPT" in " ".join($TEST_CASES) or "CPF" in " ".join($TEST_CASES)
27+
# ... Check Power Supply AND
2228

23-
Suite Setup Run Keywords
24-
... Prepare Test Suite AND
25-
... Check Power Supply
2629

2730
*** Variables ***
2831
&{TESTS_TO_RUN}= &{EMPTY}
2932
${CPU_FREQUENCY_MEASURE}= ${TRUE}
3033
${TESTS_IN_UBUNTU_SUPPORT}= ${TRUE}
3134
@{TESTED_LINUX_DISTROS}= 202 201
35+
${TEST_ID}= ${EMPTY}
3236

3337

3438
*** Test Cases ***
35-
36-
# Tests that can be performed immediately
37-
_CANARY_CPF001.201 CPU not stuck on initial frequency (Ubuntu)
38-
VAR ${TEST_ID}= CPF001.201
39-
Add Test Case Skip Condition
40-
... ${TESTS_TO_RUN}
41-
... ${TEST_ID}
42-
... not ${CPU_FREQUENCY_MEASURE}
43-
... frequency measure not supported
44-
Add Test Case Skip Condition
45-
... ${TESTS_TO_RUN}
46-
... ${TEST_ID}
47-
... not ${TESTS_IN_UBUNTU_SUPPORT}
48-
... tests in Ubuntu not supported
49-
Add Test Case Skip Condition
50-
... ${TESTS_TO_RUN}
51-
... ${TEST_ID}
52-
... '201' not in ${TESTED_LINUX_DISTROS}
53-
... Ubuntu not in tested distros
54-
Skip If Not Supported ${TEST_ID} ${TESTS_TO_RUN}
55-
56-
_CANARY_CPF005.201 CPU runs on expected frequency (Ubuntu)
57-
VAR ${TEST_ID}= CPF005.201
58-
Add Test Case Skip Condition
59-
... ${TESTS_TO_RUN}
60-
... ${TEST_ID}
61-
... not ${CPU_FREQUENCY_MEASURE}
62-
... frequency measure not supported
63-
Add Test Case Skip Condition
64-
... ${TESTS_TO_RUN}
65-
... ${TEST_ID}
66-
... not ${TESTS_IN_UBUNTU_SUPPORT}
67-
... tests in Ubuntu not supported
68-
Add Test Case Skip Condition
69-
... ${TESTS_TO_RUN}
70-
... ${TEST_ID}
71-
... '201' not in ${TESTED_LINUX_DISTROS}
72-
... Ubuntu not in tested distros
73-
Add Test Case Skip Condition
74-
... ${TESTS_TO_RUN}
75-
... ${TEST_ID}
76-
... ${LAPTOP_PLATFORM}
77-
... The Platform is a Laptop
78-
Skip If Not Supported ${TEST_ID} ${TESTS_TO_RUN}
79-
80-
_CANARY_CPT001.201 CPU temperature without load (Ubuntu)
81-
VAR ${TEST_ID}= CPT001.201
82-
Add Test Case Skip Condition
83-
... ${TESTS_TO_RUN}
84-
... ${TEST_ID}
85-
... not ${TESTS_IN_UBUNTU_SUPPORT}
86-
... tests in Ubuntu not supported
87-
Add Test Case Skip Condition
88-
... ${TESTS_TO_RUN}
89-
... ${TEST_ID}
90-
... '${ENV_ID_UBUNTU}' not in ${TESTED_LINUX_DISTROS}
91-
... Ubuntu not in tested distros
92-
Add Test Case Skip Condition
93-
... ${TESTS_TO_RUN}
94-
... ${TEST_ID}
95-
... ${LAPTOP_PLATFORM}
96-
... The Platform is a Laptop
97-
Skip If Not Supported ${TEST_ID} ${TESTS_TO_RUN}
98-
99-
10039
############################################
10140
# Tests that can be done immediately #
10241
############################################
10342
_GATHER_Background Measurements Data Init (no load) (Ubuntu)
104-
${prepare_sensors}= Will Test Be Run Regex ${TESTS_TO_RUN} CP[TF]
43+
${prepare_sensors}= Will Test Be Run Regex ${TESTS_TO_RUN} CP[TF]
10544
IF ${prepare_sensors} Prepare Sensors
10645
${check_psu}= Will Test Be Run Regex ${TESTS_TO_RUN} CP[TF]
10746
IF ${check_psu} Check Power Supply
10847

10948
_GATHER_Background Measurements Immediate (no load) (Ubuntu)
11049
# immediately skip if no tests want these measurements
111-
${will_any_be_run}= Will Test Be Run Regex ${TESTS_TO_RUN} CPF001.201
50+
${will_any_be_run}= Will Test Be Run Regex ${TESTS_TO_RUN} CPF001.201
11251
Skip If not ${will_any_be_run}
11352

11453
Power On
@@ -118,7 +57,7 @@ _GATHER_Background Measurements Immediate (no load) (Ubuntu)
11857

11958
# CPF001.201 steps
12059
VAR ${test_id}= CPF001.201
121-
${check_frequency}= Will Test Be Run ${TESTS_TO_RUN} ${test_id}
60+
${check_frequency}= Will Test Be Run ${TESTS_TO_RUN} ${test_id}
12261
IF ${check_frequency}
12362
Sleep 10s
12463
@{frequencies}= Get CPU Frequencies In Ubuntu
@@ -143,6 +82,7 @@ CPF001.201 CPU not stuck on initial frequency (Ubuntu)
14382
#############################################################################
14483
# Tests that gather measurements on Ubuntu, no load, n/a power source #
14584
#############################################################################
85+
14686
_GATHER_Background Measurements (no load) (Ubuntu)
14787
${will_any_be_run}= Will Test Be Run Regex
14888
... ${TESTS_TO_RUN}
@@ -164,34 +104,47 @@ _GATHER_Background Measurements (no load) (Ubuntu)
164104
VAR @{stab_list}= @{EMPTY}
165105

166106
# TODO temporary debug values
167-
VAR ${FREQUENCY_TEST_MEASURE_INTERVAL}= 1
168-
VAR ${TEMPERATURE_TEST_MEASURE_INTERVAL}= 1
169-
VAR ${TEMPERATURE_TEST_DURATION}= 5
170-
VAR ${FREQUENCY_TEST_DURATION}= 5
171-
VAR ${STABILITY_TEST_DURATION}= 0
172-
173-
${next_temp_time}= Set Variable If ${gather_temps} ${TEMPERATURE_TEST_MEASURE_INTERVAL} 999999
174-
${next_stab_time}= Set Variable If ${gather_temps} ${STABILITY_TEST_MEASURE_INTERVAL} 999999
175-
${next_freq_time}= Set Variable If ${gather_temps} ${FREQUENCY_TEST_MEASURE_INTERVAL} 999999
107+
VAR ${frequency_test_measure_interval}= 1
108+
VAR ${temperature_test_measure_interval}= 1
109+
VAR ${temperature_test_duration}= 5
110+
VAR ${frequency_test_duration}= 5
111+
VAR ${stability_test_duration}= 0
112+
113+
IF ${gather_temps}
114+
VAR ${next_temp_time}= ${temperature_test_measure_interval}
115+
ELSE
116+
VAR ${next_temp_time}= 999999
117+
END
118+
IF ${gather_temps}
119+
VAR ${next_stab_time}= ${STABILITY_TEST_MEASURE_INTERVAL}
120+
ELSE
121+
VAR ${next_stab_time}= 999999
122+
END
123+
IF ${gather_temps}
124+
VAR ${next_freq_time}= ${frequency_test_measure_interval}
125+
ELSE
126+
VAR ${next_freq_time}= 999999
127+
END
176128

177129
${start}= DateTime.Get Current Date
178130
${timer}= Evaluate 0
179-
VAR ${longest_duration}= max(${TEMPERATURE_TEST_DURATION}, ${FREQUENCY_TEST_DURATION}, ${STABILITY_TEST_DURATION})
131+
VAR ${longest_duration}=
132+
... max(${temperature_test_duration}, ${frequency_test_duration}, ${stability_test_duration})
180133

181134
WHILE ${timer} < ${longest_duration}
182135
${now}= Get Current Date
183136
${timer}= Subtract Date From Date ${now} ${start}
184137

185-
IF ${gather_temps} and ${TEMPERATURE_TEST_DURATION} >= ${timer} >= ${next_temp_time}
138+
IF ${gather_temps} and ${temperature_test_duration} >= ${timer} >= ${next_temp_time}
186139
${temperature}= Get CPU Temperature
187-
${next_temp_time}= Evaluate ${timer} + ${TEMPERATURE_TEST_MEASURE_INTERVAL}
140+
${next_temp_time}= Evaluate ${timer} + ${temperature_test_measure_interval}
188141
Append To List ${temp_list} ${temperature}
189142
Log To Console ${timer}s: Temperature: ${temperature}
190143
END
191144

192-
IF ${gather_freqs} and ${FREQUENCY_TEST_DURATION} >= ${timer} >= ${next_freq_time}
145+
IF ${gather_freqs} and ${frequency_test_duration} >= ${timer} >= ${next_freq_time}
193146
${freqs}= Get CPU Frequencies In Ubuntu
194-
${next_freq_time}= Evaluate ${timer} + ${FREQUENCY_TEST_MEASURE_INTERVAL}
147+
${next_freq_time}= Evaluate ${timer} + ${frequency_test_measure_interval}
195148
Append To List ${freq_list} ${freqs}
196149
Log To Console ${timer}s: Frequencies: ${freqs}
197150
END
@@ -236,6 +189,7 @@ CPF005.201 CPU runs on expected frequency (Ubuntu)
236189
END
237190
END
238191

192+
239193
*** Keywords ***
240194
Wait Until Time
241195
[Arguments] ${target}
@@ -244,19 +198,20 @@ Wait Until Time
244198
${sleep}= Evaluate max(min(${diff}, 0.1), 0.01)
245199
Sleep ${sleep}
246200

247-
Add Test Case Skip Condition
248-
[Arguments] ${tests_dict} ${test_id} ${condition} ${skip_reason}
249-
${current_details}= Get From Dictionary ${tests_dict} ${test_id} default=${FALSE}
201+
Add Skip Condition
202+
[Arguments] ${condition} ${skip_reason}
203+
# [Globals] ${TESTS_TO_RUN} ${TEST_ID}
204+
${current_details}= Get From Dictionary ${TESTS_TO_RUN} ${TEST_ID} default=${FALSE}
250205
IF not ${current_details}
251-
VAR &{support_details}= test=${test_id} run=${TRUE} reason=supported
206+
VAR &{support_details}= test=${TEST_ID} run=${TRUE} reason=supported
252207
ELSE
253208
VAR ${support_details}= ${current_details}
254209
END
255210
${run}= Get From Dictionary ${support_details} run
256211
IF ${run} and (${condition})
257212
Set To Dictionary ${support_details} run=${FALSE} reason=${skip_reason}
258213
END
259-
Set To Dictionary ${tests_dict} ${test_id}=${support_details}
214+
Set To Dictionary ${TESTS_TO_RUN} ${TEST_ID}=${support_details}
260215

261216
Skip If Not Supported
262217
[Arguments] ${test_id} ${tests_dict}
@@ -273,7 +228,6 @@ Will Test Be Run
273228
${value}= Get From Dictionary ${test} run ${FALSE}
274229
RETURN ${value}
275230

276-
277231
Get Test Value
278232
[Arguments] ${tests_dict} ${test} ${key}
279233
${test}= Get From Dictionary ${tests_dict} ${test}
@@ -295,7 +249,6 @@ Set Test Outputs
295249
[Arguments] ${tests_dict} ${test} ${value}
296250
Set Test Value ${tests_dict} ${test} outputs ${value}
297251

298-
*** Keywords ***
299252
Will Test Be Run Regex
300253
[Arguments] ${dict} ${pattern}
301254
FOR ${key} IN @{dict.keys()}
@@ -306,4 +259,29 @@ Will Test Be Run Regex
306259
IF ${run} RETURN ${TRUE}
307260
END
308261
END
309-
RETURN ${FALSE}
262+
RETURN ${FALSE}
263+
264+
Prepare Parallel Test Suite
265+
# Preparing parallel test cases
266+
VAR ${TEST_ID}= CPF001.201 scope=TEST
267+
Add Skip Condition not ${CPU_FREQUENCY_MEASURE} frequency measure not supported
268+
Add Skip Condition not ${TESTS_IN_UBUNTU_SUPPORT} tests in Ubuntu not supported
269+
Add Skip Condition '201' not in ${TESTED_LINUX_DISTROS} Ubuntu not in tested distros
270+
271+
VAR ${TEST_ID}= CPF005.201 scope=TEST
272+
Add Skip Condition not ${CPU_FREQUENCY_MEASURE} frequency measure not supported
273+
Add Skip Condition not ${TESTS_IN_UBUNTU_SUPPORT} tests in Ubuntu not supported
274+
Add Skip Condition '201' not in ${TESTED_LINUX_DISTROS} Ubuntu not in tested distros
275+
Add Skip Condition ${LAPTOP_PLATFORM} The Platform is a Laptop
276+
277+
VAR ${TEST_ID}= CPT001.201 scope=TEST
278+
Add Skip Condition not ${TESTS_IN_UBUNTU_SUPPORT} tests in Ubuntu not supported
279+
Add Skip Condition '${ENV_ID_UBUNTU}' not in ${TESTED_LINUX_DISTROS} Ubuntu not in tested distros
280+
Add Skip Condition ${LAPTOP_PLATFORM} The Platform is a Laptop
281+
282+
Log To Console Tests to run:
283+
FOR ${key} IN @{TESTS_TO_RUN.keys()}
284+
${test}= Get From Dictionary ${TESTS_TO_RUN} ${key}
285+
${run}= Get From Dictionary ${test} run
286+
IF ${run} Log To Console ${key}
287+
END

0 commit comments

Comments
 (0)