Skip to content

Continuous releases #810

Closed
Closed

Description

@ben

Looks like Travis has the capability of continuous deployment, which we could use to always be pushing fresh builds to GitHub releases. Some requirements:

  • Only run on changes to master
  • Tag as 2.1.NNNN (we don't care how big the numbers get, but they should be in the 2.1 series)
  • Publish epub, mobi, and PDF binaries
  • Update the git-scm site to provide download links (to the newest version)

Activity

  1. jnavila commented on Sep 22, 2017

    @jnavila
    Member

    The thing is I don't have administrative rights on the repo, only push, even for the French translation. Do you have these rights? If so, could you share them at least for the French repo?

    Also, as you mentionned, that would require tagging every time we would need a new release. What would be the policy?

  2. ben commented on Sep 22, 2017

    @ben
    MemberAuthor

    Permissions granted on progit2-fr.

    I think just a monotonically-increasing patch number, that gets bumped every time a commit lands on master would work okay. I don't mind the versions going up to 2.1.173. I think Travis can create a tag on its own?

  3. jnavila commented on Sep 22, 2017

    @jnavila
    Member

    Travis can only tag for npm, it seems. I don't know if we can make a hook on github to tag after each push. Another question is whether Github accepts that we link the files to download from git-scm.com. Yet another question is, what do we do with translations?

  4. ben commented on Sep 22, 2017

    @ben
    MemberAuthor

    Looks like the releases API lets you upload your own binaries. And if a direct link to an asset doesn't work when the referrer isn't github.com, maybe a link to the release itself would?

    Translation repos could work the same way, managing their own binary releases. We'd need to update git-scm.com to link to them as well.

  5. jnavila commented on Sep 23, 2017

    @jnavila
    Member

    @peff What do you think of automatic deployment?

  6. peff commented on Sep 24, 2017

    @peff

    Automatic deployment sounds great. We should trust the translation teams to manage their repos and keep their master branch at something sane.

    Right now all of the updates to git-scm.com happen through nightly scheduled jobs. It probably wouldn't be too hard to hit the releases API for each translation, checking for new links.

    Direct-linking to uploaded assets should be fine, I think.

  7. jtcmedia commented on Oct 24, 2017

    @jtcmedia

    Were you all in need of a Travis CI file? I'm already am using one on my fork https://github.com/jtcmedia/progit2 if this is of any use.

  8. ben commented on Oct 25, 2017

    @ben
    MemberAuthor

    I just pushed that, and it looks like Travis will build whenever we push a tag. So that takes care of the first and third bullet points above.

  9. jnavila commented on Oct 25, 2017

    @jnavila
    Member

    I already have a fully automated build+tag+deploy on my personal repo.

    https://github.com/jnavila/progit2-fr/blob/3af9c11cb8f04320f4a0f40a4d5c7bc8c9f00b14/.travis.yml

    This version needs some more love:

    • describe the setup for uploading the artefacts
    • generalize the script (there are still some hard references to the repo)

    This version needs more administration for the setup, which is one of my major concerns for translations.

    I'll try it on progit-fr first (hopefully soon) but you can already try the tagging part.

  10. ben commented on Oct 25, 2017

    @ben
    MemberAuthor

    OK, I got this working. It involved updating .travis.yml, writing a script, and generating a new token for the GitHub API, but it works now. Here's a Travis build from a push to master which pushed the 2.1.2 tag, and here's the https://travis-ci.org/progit/progit2/builds/292755132 that published the release.

    Going to close this out for now, since everything actionable in this repo is done. The one thing left is for git-scm.com's nightly build to look at the releases and pick the latest one to link to.

  11. jnavila commented on Oct 25, 2017

    @jnavila
    Member

    Sorry to bother, but your version of tagging is clearly ligher than mine. Could you rapidly describe the steps, so that they can be reproduced in other repos?

  12. ben commented on Oct 25, 2017

    @ben
    MemberAuthor

    Sure. Here's where we compute the next patch number:

    LASTPATCH=$(git describe --tags | cut -d- -f1 | cut -d. -f3)
    PATCH=$(($LASTPATCH+1))

    git describe --tags usually outputs something like 2.1.5-11-g608d7fa. We pipe this to cut, which splits it using hyphens and takes the first result (2.1.5 here), then we cut again, this time splitting by .s and grabbing the third one (5 here). The next line just increments by one.

    Here's where we apply the tag:

    curl -H "Authorization: token $GITHUB_KEY" \
         -X POST \
         -d "{\"ref\":\"refs/tags/2.1.$PATCH\", \"sha\":\"$TRAVIS_COMMIT\"}" \
         https://api.github.com/repos/progit/progit2/git/refs

    This just uses the GitHub api for refs to create a tag. I created a new access token and did travis encrypt to generate the string you see in .travis.yml using GITHUB_KEY=<key> as the input.

    Here's the JSON payload we send to GitHub:

    {
      "ref": "refs/tags/2.1.6",
      "sha": "098963b50060aebaaf5091779cba78306d1c6e37"
    }

    Which creates a new tag. That triggers the webhook that Travis has installed, which then runs the build again on that commit and deploys the result to a release. Does that make sense?

  13. jtcmedia commented on Oct 25, 2017

    @jtcmedia

    That's slick @ben. Thanks for posting!

  14. YueLinHo commented on Oct 26, 2017

    @YueLinHo
    Contributor

    So... will be a lot of tag?

  15. ben commented on Oct 26, 2017

    @ben
    MemberAuthor

    Yes, this will unfortunately result in a lot of tags. I wonder what happens if we remove the tags once the deploy is done…

  16. ben commented on Oct 26, 2017

    @ben
    MemberAuthor

    I just manually deleted the 2.1.1 tag, and it turned that GitHub release into a "draft." I don't think that's what we want, so we'll just be accumulating tags. I'm okay with that trade off.

    If someone else isn't, it would be possible to rewrite my tag_on_master script to upload built binaries and create a release without adding the tag. I don't have time to tinker with it just now.

  17. jnavila commented on Oct 26, 2017

    @jnavila
    Member

    I think we can do something about tidying automatically with octokit

  18. taichunmin commented on Dec 21, 2017

    @taichunmin

    It is pretty easier to use GitLab to auto build the book.
    I use GitLab CI + docker to build book, then push to GitLab Pages.
    Doesn't require useless tags.

    https://gitlab.com/taichunmin/progit2-zh-tw/

    https://taichunmin.gitlab.io/progit2-zh-tw/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions