Feature/CSPL-4397 Eliminate reliance on time in tests

[kasiakoziol]

Description

It removes time dependencies from the the tests (integration, smoke).

Key Changes

• Replacing hard time.Sleep(...) with condition-based Eventually(...)
• Replacing long "debug sleeps" in App Framework tests with explicit "wait until phase/event happens"
• Introducing/standardizing polling helpers
• Defining separate poll intervals and introducing a specific timeout for app installation
• Making scale event emission depend on "ready replica convergence"
• Replacing Monitoring Console config map checks “sleep then assert” with “wait until config map contains expected entries”

Testing and Verification

Automated tests

Related Issues

Jira: https://splunk.atlassian.net/browse/CSPL-4397

PR Checklist

• Code changes adhere to the project's coding standards.
• Relevant unit and integration tests are included.
• Documentation has been updated accordingly.
• All tests pass locally.
• The PR description follows the project's guidelines.

[github-actions]

CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅

[patryk]

Please configure your bot to stop tagging me in the PRs to this project. Thanks.

[patrykw-splunk]

@patryk Our apologies for that. We'll fix it asap.
PS. great GH alias :)

[patryk]

PS. great GH alias :)

Heh, I suppose the perk of signing up for niche (in 2009) sites early.

[coveralls]

Pull Request Test Coverage Report for Build 22764325242

Details

• 22 of 37 (59.46%) changed or added relevant lines in 3 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage increased (+0.07%) to 84.005%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
pkg/splunk/enterprise/indexercluster.go	7	12	58.33%
pkg/splunk/enterprise/searchheadclusterpodmanager.go	7	12	58.33%
pkg/splunk/enterprise/standalone.go	8	13	61.54%

Totals
Change from base Build 22743106510:	0.07%
Covered Lines:	11848
Relevant Lines:	14104

---

💛 - Coveralls