feat: Playwright scenario + load test for SF2.0#3004
Merged
thecristen merged 8 commits intomainfrom Mar 12, 2026
Merged
Conversation
lvachon1
reviewed
Mar 11, 2026
lvachon1
approved these changes
Mar 11, 2026
Contributor
lvachon1
left a comment
lvachon1
left a comment
There was a problem hiding this comment.
The tests all run and seem to work well, however I think the chooseURL function isn't randomizing it's choices
adding this avoids Green-(B|C|D|E) linking to Green (no branch) # Conflicts: # lib/dotcom_web/live/schedule_finder_live.ex
thecristen
force-pushed
the
cbj/sf2.0/load-testing
branch
from
fb67bf9 to
3b9cb2d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Scope
Asana Ticket: Add basic integration test for SF2.0 and Add and run load tests for SF2.0
Implementation
We're re-using our existing test setups to add tests for our new departures!
integration/scenarios/gets a new Playwright test --view-departures.js. This is a brief test which loads a randomly selected departure page, checking for a successful response and some expected text and links. As part ofscenarios/it'll also get run by our monitor, which runs and reports on all scenarios many times per day.load_tests/departures.ymlspecifies a new load test using Artillery and its Playwright engine, which lets us write the test code with Playwright. Viadepartures.js, it references the newscenarios/view-departures.js, expanding upon it with additional actions and checks - opening an upcoming departure, changing the selected schedule, etc. The YAML config can be changed ad-hoc to change the rate/amount of visits in the load test.integration/e2e_tests/gets a new Playwright test --visit-departures.spec.js. This mostly just runs the aforementioned load test code, but using the Playwright test runner.schedule_finder_visits.csv, which I extracted from a Splunk query tabulating visits to our existing Schedule Finder on Tuesday, March 3rd by distinct route, direction, and stop. I also wrote some code which parses this and can return a/departuresURL, randomly generated but weighted by actual visits in the CSV.It was a bit insane trying to get interdependent scripting bits to play nice together across the CommonJS/ES module systems (seems like either Playwright and/or Artillery each had stringent expectations). This setup was the best I could come up with which still worked without my duplicating test code between the two systems.
Screenshots
Testing looks something like that:
How to test
If you haven't already, run
npm install --ignore-scriptsfrom the root folder.These run against localhost if you don't specify a
--targetorHOST.The briefer Playwright scenario:
The more elaborate Playwright test. I attempted to make it flexible enough such that it shouldn't break on a route/direction/stop which doesn't happen to be in service at the tested moment.
I recommend learning the CLI. Can run these with
--uito run the test from a fun interface, with--repeat-each=50to run it many times, etc.(Running the Playwright tests against another server may have varied results, since this branch has a few HTML fixes on which the test cases depend.)
The load tests