If in doubt discuss with other developers in the #dm-development Slack channel
This manual page describes the day to day work of a developer on the Digital Marketplace, much of which will be writing and reviewing pull requests. It supplements (and in some cases supersedes) the GDS Way, particular the pages on how to store source code, how to review code, and the page on pull requests in general.
So, you’re looking at the ticket at the top of your team’s backlog and are about to start work building out the feature…
Before even thinking about beginning to touch a line of code, do a story kickoff. Talk to anyone on the team who might have an opinion about the technical implementation or is likely to be affected by changes to the interface and hash out what the solution to your particular problem might look like. A list of interested people may be in the story. By the time you claim the story on Trello it should be abundantly clear what your goals are.
We develop all our code on Git feature branches and review these via GitHub pull requests.
When developing locally, apps can be configured to talk to deployed versions of the services. This is configured using environment variables. The default configuration points at local versions of the API and Search API (set in config.py), but these can be overridden.
For example, see the buyer app.
When development of a feature is complete and the application level tests pass locally open a pull request. This will cause Travis CI to run run the application tests and update the pull request. From this point on the developer who opened the pull request is responsible for ensuring the feature moves through the deployment process.
As soon as you’re happy with the pull request description it’s time to request review. Post a link to the pull request in the #dm-review channel. All developers on the Digital Marketplace should be a member of this channel and watch it regularly for new pull requests, but if someone doesn’t look at your pull request by the end of the day it will be reposted by the alphagov seal bot at the beginning of the next day. If your pull request needs to be urgently reviewed make sure to mention that when posting the link to the pull request.
All developers should be involved in reviewing pull requests (although only one approval is needed before merging).
It isn’t expected that everyone reads every pull request, but everyone should be spending a small amount of time each day reviewing. Don’t be shy about reviewing code you’re unfamiliar with - it’s a good time to ask questions and learn about the broader codebase. However make sure you understand the changes before approving.
We review pull requests with the review tool on GitHub. Discussion, questions, and constructive feedback are all encouraged and expected; also don’t forget to leave friendly emojis!
Reviewers may approve your pull request without comment, approve it with a comment and expect you to action the comment before merging (trusting that you will), or leave comments without approving if they have unresolved questions or concerns.
You can update your pull request as many times as you want before merging, although you might want to avoid making big changes after it has been approved. You should use git rebase liberally to keep the commits based on the current master branch and the history clean (the exception to this is when pairing on a pull request, when it’s better to wait until it’s ready for merging before rebasing).
Once your reviewers have reviewed the pull request and are happy with the proposed changes they will approve it using the GitHub review tool. This indicates that you’re clear to merge it when you’re ready. It is not easy to change a pull request once merged, so at the point of merging a pull request everyone implicated should agree on the implemented solution.
Merging a pull request on one of the main Flask applications will automatically kick off a series of steps involved in deploying to preview:
The application is tagged with the pull request that caused the build
An artifact is built for this tag and deployed to preview.
Once the deployment is complete a run of the functional and visual regression tests will be started.
The build tagging will fail if, when merging, you change the merge commit message. This is because the deployment process will not be able to regex out the PR number from the commit message.
Once the application has been deployed to preview, the smoke tests, functional tests and visual regression tests will be run automatically. After they have both passed it is ready to be viewed on preview.
You should consider doing some manual testing of your own when a change has been made on preview. For larger changes and new features you may also want to take some screenshots to share with your team’s product manager, or have them go through the journey themselves.
Larger changes should ideally have feature tests and visual regression tests added to the respective suites.
Preview can be found at: https://www.preview.marketplace.team/
Deploying to staging is the first step of deploying a feature to production. Our staging environment acts as a backstop to catch any breaking changes before deploying to production; usually only one change from production should be present in staging at a time, and not for very long.
Before deploying to staging or production the feature must have been OK’d for release by doing some manual testing and discussing with the product manager if necessary.
Because of limitations of our current release process, the longer your change sits in preview without being deployed, the more likely it will be that you will be unable to release it without also releasing another’s developers changes.
Be sure to check that all changes going out in your release have been OK’d. We have a lag radiator that can help with
this, it shows how many merges are undeployed for each app. Clicking on the name for each app links to a GitHub page
with all undeployed commits. To access the lag radiator decrypt digitalmarketplace-credentials/links.yaml and
deploy_lag_radiator URL. Please don’t share it in plaintext, as it contains an API key.
When you’re ready to go take the Gorilla of Deploy - YOLO - and keep it close!
Check the channel topic in the #dm-release Slack channel indicates it is currently free.
If the Deploy Lag Radiator shows there are other team members’ commits included in the deploy; tag them in a message in the #dm-release Slack channel; so they are aware their changes are about to be deployed.
Update the channel topic in the #dm-release Slack channel to indicate you are deploying.
Find the relevant release pipeline job for the app you’re releasing in Jenkins, hover your mouse pointer over the “release to staging” box and click “Proceed”. This will take the artefact for the version of the application currently deployed into preview and deploy it to staging.
Once deployment to staging is complete a run of the functional tests will be started - you will be notified of the result in the #dm-release Slack channel once it completes. You should see these tests pass before promoting any changes to production. If tests fail either fix the tests or fix your code before taking it any further.
For high risk changes or changes you could not manually test on preview you may also want to do manual testing on staging. Once you’ve deployed you can check your work at: https://www.staging.marketplace.team/.
You can view its status at the
_status endpoint, e.g. https://www.staging.marketplace.team/suppliers/_status for the
Once the staging functional tests have successfully passed and you’re happy to go to production, visit the same pipeline job in Jenkins, hover your mouse pointer over the “release to production” box and click “Proceed”.
This will deploy the artefact currently in staging into production.
You should now check the feature works as expected on the live site. Once we’re happy the feature is live it is DONE!
Release YOLO back into the wild.
Update the channel topic in the #dm-release Slack channel to indicate it is free.