Skip to content

check-suspend-resume: check /sys/power/suspend_stats/*fail* at every step + dump much more /sys/ info - #900

Merged
marc-hb merged 3 commits into
thesofproject:mainfrom
marc-hb:suspend-stats
May 11, 2022
Merged

marc-hb merged 3 commits into
thesofproject:mainfrom
marc-hb:suspend-stats

Conversation

@marc-hb

@marc-hb marc-hb commented May 6, 2022 •

Copy link
Copy Markdown
Collaborator

Implements #898 and a lot more. 3 commits.

@marc-hb
marc-hb requested a review from plbossart May 6, 2022 00:15
@marc-hb

marc-hb commented May 6, 2022

Copy link
Copy Markdown
Collaborator Author

https://sof-ci.01.org/softestpr/PR900/build18/devicetest reveals that wakeup_count often bumps by 2 instead of 1... any idea why?

cc: @plbossart , @ujfalusi , @kv2019i , @ranj063

@ujfalusi

ujfalusi commented May 6, 2022

Copy link
Copy Markdown
Contributor

@marc-hb, and in some cases the wakup_count is not incremented?
https://sof-ci.01.org/softestpr/PR900/build18/devicetest/?model=TGLU_UP_HDA_ZEPHYR&testcase=check-suspend-resume-5

Can you print the wakeup_count along with the files from suspend_stats?

@marc-hb

marc-hb commented May 6, 2022

Copy link
Copy Markdown
Collaborator Author

Can you print the wakeup_count along with the files from suspend_stats?

I will "downgrade" this PR and print it instead of failing the test.

@marc-hb marc-hb changed the title check-suspend-resume: check /sys/power/suspend_stats/*fail* at every step check-suspend-resume: check /sys/power/suspend_stats/*fail* at every step + dump much more /sys/ info May 6, 2022
@marc-hb
marc-hb marked this pull request as ready for review May 7, 2022 00:02
@marc-hb
marc-hb requested a review from a team as a code owner May 7, 2022 00:02
@marc-hb

marc-hb commented May 7, 2022

Copy link
Copy Markdown
Collaborator Author

I will "downgrade" this PR and print wrong wakeup_counts instead of failing the test.

Done in v2 + dumping some additional info from /sys.

The two failures in https://sof-ci.01.org/softestpr/PR900/build19/devicetest/ are known and unrelated.

The wakeup_count is much more unreliable than I expected. It seems platform-dependent.

Other values seems fine.

Please help review and merge this so we can start collecting more data - especially on systems that crash or timeout.

Comment thread test-case/check-suspend-resume.sh Outdated
# set up checkpoint for each iteration
setup_kernel_check_point
sleep_count=$(cat /sys/power/wakeup_count)
: $((expected_wakeup_count++))

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no idea what this does.

using 'advanced sh' will restrict the circle of reviewers.

we need the equivalent of 'simple english' in wikipedia for shell scripts, something that everyone can understand.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replaced with expected_wakeup_count=$((expected_wakeup_count+1))

Comment thread test-case/check-suspend-resume.sh Outdated
{
printf '\n\n'

INITIAL_DBG_SOURCES="$LOG_ROOT"/intial_wakeup_sources.txt

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

initial_wakeup_sources.txt

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, thx!

marc-hb added 3 commits May 9, 2022 09:39
"greater or equal" has never ever reported any error. Test for the
expected value instead but print only a warning for now because of the
high number of failures this causes, example:

  https://sof-ci.01.org/softestpr/PR900/build18/devicetest/

Also catch any unexpected increase while the system is up - trust no
one.

Fix the error message to report exactly what was found, not a guess.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
As requested in thesofproject#898

Also dumps a lot more system information.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
@marc-hb

marc-hb commented May 9, 2022

Copy link
Copy Markdown
Collaborator Author

For https://sof-ci.01.org/softestpr/PR900/build20/devicetest/ CML_SKU0955_HDA was not available. https://sof-ci.01.org/softestpr/PR900/build20/devicetest/?model=WHL_UPEXT_HDA_ZEPHYR&testcase=check-suspend-resume-with-playback-5 timed out with more logs than usual which is exactly the purpose of this PR.

@greg-intel greg-intel left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

dlogi "Check for the kernel log status"
# check kernel log for each iteration to catch issues
sof-kernel-log-check.sh "$KERNEL_CHECKPOINT" || die "Caught error in kernel log"
sof-kernel-log-check.sh "$KERNEL_CHECKPOINT" || dump_and_die "Caught error in kernel log"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should line 146 be split across two lines as well?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

93 characters so this is over the traditional 80 limit but below Linux' new, official 100 characters limit...

@marc-hb

marc-hb commented May 11, 2022

Copy link
Copy Markdown
Collaborator Author

Thanks @greg-intel for the review. I will merge this tomorrow if there is no other review.

@marc-hb
marc-hb requested a review from lyakh May 11, 2022 05:00
@marc-hb
marc-hb merged commit 453b0af into thesofproject:main May 11, 2022
@marc-hb
marc-hb deleted the suspend-stats branch May 11, 2022 17:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants