Development and deployment process

Tip

If in doubt discuss with other developers in the #dm-development Slack channel

This manual page describes the day to day work of a developer on the Digital Marketplace, much of which will be writing and reviewing pull requests. It supplements (and in some cases supersedes) the GDS Way, particular the pages on how to store source code, how to review code, and the page on pull requests in general.

Story Kickoff

So, you’re looking at the ticket at the top of your team’s backlog and are about to start work building out the feature…

Before even thinking about beginning to touch a line of code, do a story kickoff. Talk to anyone on the team who might have an opinion about the technical implementation or is likely to be affected by changes to the interface and hash out what the solution to your particular problem might look like. A list of interested people may be in the story. By the time you claim the story on Trello it should be abundantly clear what your goals are.

Develop locally on feature branches

We develop all our code on Git feature branches and review these via GitHub pull requests.

When developing locally, apps can be configured to talk to deployed versions of the services. This is configured using environment variables. The default configuration points at local versions of the API and Search API (set in config.py), but these can be overridden.

For example, see the buyer app.

Open a pull request

When development of a feature is complete and the application level tests pass locally open a pull request. This will cause Travis CI to run run the application tests and update the pull request. From this point on the developer who opened the pull request is responsible for ensuring the feature moves through the deployment process.

If your team uses Trello then paste the link to the corresponding card into the pull request message. The GitHub Trello poster will then add a link to your pull request to the Trello card for you 1.

1

To set up the GitHub Trello poster you need to add a webhook to the repo and add the GTP user to the Trello board. See the GitHub Trello poster README for details.

Request review

As soon as you’re happy with the pull request description it’s time to request review. Post a link to the pull request in the #dm-review channel. All developers on the Digital Marketplace should be a member of this channel and watch it regularly for new pull requests, but if someone doesn’t look at your pull request by the end of the day it will be reposted by the alphagov seal bot at the beginning of the next day. If your pull request needs to be urgently reviewed make sure to mention that when posting the link to the pull request.

Note

All developers should be involved in reviewing pull requests (although only one approval is needed before merging).

It isn’t expected that everyone reads every pull request, but everyone should be spending a small amount of time each day reviewing. Don’t be shy about reviewing code you’re unfamiliar with - it’s a good time to ask questions and learn about the broader codebase. However make sure you understand the changes before approving.

We review pull requests with the review tool on GitHub. Discussion, questions, and constructive feedback are all encouraged and expected; also don’t forget to leave friendly emojis!

Reviewers may approve your pull request without comment, approve it with a comment and expect you to action the comment before merging (trusting that you will), or leave comments without approving if they have unresolved questions or concerns.

You can update your pull request as many times as you want before merging, although you might want to avoid making big changes after it has been approved. You should use git rebase liberally to keep the commits based on the current master branch and the history clean (the exception to this is when pairing on a pull request, when it’s better to wait until it’s ready for merging before rebasing).

Merge

Once your reviewers have reviewed the pull request and are happy with the proposed changes they will approve it using the GitHub review tool. This indicates that you’re clear to merge it when you’re ready. It is not easy to change a pull request once merged, so at the point of merging a pull request everyone implicated should agree on the implemented solution.

Merging a pull request on one of the main Flask applications will automatically kick off a series of steps involved in deploying to preview:

  1. The application is tagged with the pull request that caused the build release-<PR number>.

  2. An artifact is built for this tag and deployed to preview.

  3. Once the deployment is complete a run of the functional and visual regression tests will be started.

Warning

The build tagging will fail if, when merging, you change the merge commit message. This is because the deployment process will not be able to regex out the PR number from the commit message.

Working on shared repositories

Several of our repositories contain shared code used by our different applications:

When updating any of these we need to publish a new version (tagged with a new version number) when we want to share changes. So before making a pull-request with changes in one of these repositories:

  • Update the version number number on your working branch – the location of this varies depending on the repository you’re working with.

    • The frameworks and frontend-toolkit repos both define the version as part of their package.json manifest.

    • The apiclient, content-loader and utils repos have an __init__.py file inside of their module folder.

  • Once your pull request bumping the version is merged, Jenkins will automatically run the push-tag.sh script. This will release the new version with the version number you supplied, thus making it available to import in the main apps.

Test on preview

Once the application has been deployed to preview, the smoke tests, functional tests and visual regression tests will be run automatically. After they have both passed it is ready to be viewed on preview.

You should consider doing some manual testing of your own when a change has been made on preview. For larger changes and new features you may also want to take some screenshots to share with your team’s product manager, or have them go through the journey themselves.

Note

Larger changes should ideally have feature tests and visual regression tests added to the respective suites.

Preview can be found at: https://www.preview.marketplace.team/

Deploy to staging

Deploying to staging is the first step of deploying a feature to production. Our staging environment acts as a backstop to catch any breaking changes before deploying to production; usually only one change from production should be present in staging at a time, and not for very long.

Before deploying to staging or production the feature must have been OK’d for release by doing some manual testing and discussing with the product manager if necessary.

Note

Because of limitations of our current release process, the longer your change sits in preview without being deployed, the more likely it will be that you will be unable to release it without also releasing another’s developers changes.

Be sure to check that all changes going out in your release have been OK’d. We have a lag radiator that can help with this, it shows how many merges are undeployed for each app. Clicking on the name for each app links to a GitHub page with all undeployed commits. To access the lag radiator decrypt digitalmarketplace-credentials/links.yaml and bookmark the deploy_lag_radiator URL. Please don’t share it in plaintext, as it contains an API key.

When you’re ready to go take the Gorilla of Deploy - YOLO - and keep it close!

Check the channel topic in the #dm-release Slack channel indicates it is currently free.

If the Deploy Lag Radiator shows there are other team members’ commits included in the deploy; tag them in a message in the #dm-release Slack channel; so they are aware their changes are about to be deployed.

Update the channel topic in the #dm-release Slack channel to indicate you are deploying.

Find the relevant release pipeline job for the app you’re releasing in Jenkins, hover your mouse pointer over the “release to staging” box and click “Proceed”. This will take the artefact for the version of the application currently deployed into preview and deploy it to staging.

Once deployment to staging is complete a run of the functional tests will be started - you will be notified of the result in the #dm-release Slack channel once it completes. You should see these tests pass before promoting any changes to production. If tests fail either fix the tests or fix your code before taking it any further.

For high risk changes or changes you could not manually test on preview you may also want to do manual testing on staging. Once you’ve deployed you can check your work at: https://www.staging.marketplace.team/.

You can view its status at the _status endpoint, e.g. https://www.staging.marketplace.team/suppliers/_status for the supplier frontend.

Deploy to production

Once the staging functional tests have successfully passed and you’re happy to go to production, visit the same pipeline job in Jenkins, hover your mouse pointer over the “release to production” box and click “Proceed”.

This will deploy the artefact currently in staging into production.

You should now check the feature works as expected on the live site. Once we’re happy the feature is live it is DONE!

Release YOLO back into the wild.

Update the channel topic in the #dm-release Slack channel to indicate it is free.