You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I've set up an organization templateflow at GIN, @yarikoptic please let me know if you want me to add your username to it (if so, tell me your username over there, please).
I'll start today figuring out the transfer/addition of GIN - I may even send a PR to datalad to handle GIN siblings !
However, if you datalad install that dataset and then try to datalad get, some resources fail with:
[ERROR ] not available; (Note that these git remotes have annex-ignore set: gin origin) [get(/home/oesteban/datalad/templateflow/tpl-MNI152NLin6Asym/tpl-MNI152NLin6Asym_res-02_desc-b0_mask.nii.gz)]
get(error): /home/oesteban/datalad/templateflow/tpl-MNI152NLin6Asym/tpl-MNI152NLin6Asym_res-02_desc-b0_mask.nii.gz (file) [not available; (Note that these git remotes have annex-ignore set: gin origin)]
Another interesting thing that happened is that resources that did download, were pulled [from web...] (which I guess should say [from gin...] if I had set up the sibling correctly.
I'm now trying to configure the sibling correctly for tpl-MNI152NLin2009cAsym, which is one of the first datasets where I configured GIN:
$ datalad siblings configure -d . --name gin-update --pushurl git@gin.g-node.org:/templateflow/tpl-MNI152NLin2009cAsym.git --url https://gin.g-node.org/templateflow/tpl-MNI152NLin2009cAsym --as-common-datasrc gin
CommandError: 'git -c diff.ignoreSubmodules=none annex initremote gin type=git location=https://gin.g-node.org/templateflow/tpl-MNI152NLin2009cAsym autoenable=true -c annex.dotfiles=true -c 'remote.gin-update.annex-ssh-options=-o ControlMaster=auto -S /Users/oesteban/Library/Caches/datalad/sockets/40d58c92' -c annex.retry=3 -c 'remote.gin-update.annex-ssh-options=-o ControlMaster=auto -S /Users/oesteban/Library/Caches/datalad/sockets/40d58c92'' failed with exitcode 1 under /Users/oesteban/tmp/templateflow/tpl-MNI152NLin2009cAsym
git-annex: There is already a special remote named "gin". (Use enableremote to enable an existing special remote.)
So, the question is - once you've made the mistake of setting the HTTP URL with .git suffix, how do you fix it so that, when you datalad clone (or git clone in this case, which is done automatically by the GitHub action), the proper remote is set up?
It's POST /org/:org/repos. Need to play with whether one of these orgs should be the name of the org or if it should be part of the repo name in the post data.
FWIW It is GIN days seems to me. @mih is also working on create-sibling-gogsdatalad/datalad#5935
which is confusing and worrisome (that URL pointing at gogs)
IMHO it is ok, since gin is just a fork of gogs, and I guess all of the API end points are coming from gogs
So, the question is - once you've made the mistake of setting the HTTP URL with .git suffix, how do you fix it so that, when you datalad clone (or git clone in this case, which is done automatically by the GitHub action), the proper remote is set up?
git remote remove gin (to remove it from .git/config)
git annex enableremote gin location=https://gin.g-node.org/lnnrtwttkhn/highspeed-decoding - which would fix up remote.log entry for it with the new location within git-annex branch and add that remote back to known to git.
Activity
oesteban commentedon Aug 24, 2021
Yes, OSF is becoming tricky :(
oesteban commentedon Aug 24, 2021
It seems repositories must be created manually :(
http://handbook.datalad.org/en/latest/basics/101-139-gin.html#using-gin-as-a-data-source-behind-the-scenes
yarikoptic commentedon Aug 24, 2021
FWIW, didn't try, but there is gogs (which is what
ginis based on) end-point for creating new repos: https://github.com/gogs/docs-api/tree/master/Repositories#create . We just seems didn't bother creating somecreate-sibling-gogs(or-gin) yet at "datalad level".effigies commentedon Aug 24, 2021
Was able to create https://gin.g-node.org/markiewicz/tpl-MyTestTemplate with:
tokens/gin.headeris a file of the formWhere the token was created at: https://gin.g-node.org/user/settings/applications
oesteban commentedon Aug 26, 2021
I can confirm that worked @effigies.
I've set up an organization
templateflowat GIN, @yarikoptic please let me know if you want me to add your username to it (if so, tell me your username over there, please).I'll start today figuring out the transfer/addition of GIN - I may even send a PR to datalad to handle GIN siblings !
oesteban commentedon Aug 26, 2021
@yarikoptic - I followed the handbook's instructions to add gin as a data source (https://handbook.datalad.org/en/latest/basics/101-139-gin.html#using-gin-as-a-data-source-behind-the-scenes)
It seems that annexed files are stored at GIN too - e.g., https://gin.g-node.org/templateflow/tpl-MNI152NLin6Asym
However, if you
datalad installthat dataset and then try todatalad get, some resources fail with:Another interesting thing that happened is that resources that did download, were pulled
[from web...](which I guess should say[from gin...]if I had set up the sibling correctly.What am I missing?
yarikoptic commentedon Aug 26, 2021
Make sure that you don't have .git suffix in the url for gin, ie https://gin.g-node.org/templateflow/tpl-MNI152NLin6Asym is ok, https://gin.g-node.org/templateflow/tpl-MNI152NLin6Asym.git is not.
Edit: ref datalad/datalad#5144 (comment) and there in
oesteban commentedon Aug 27, 2021
Indeed that fixed the issue.
I'm now trying to configure the sibling correctly for
tpl-MNI152NLin2009cAsym, which is one of the first datasets where I configured GIN:So, the question is - once you've made the mistake of setting the HTTP URL with
.gitsuffix, how do you fix it so that, when youdatalad clone(orgit clonein this case, which is done automatically by the GitHub action), the proper remote is set up?oesteban commentedon Aug 27, 2021
@effigies - have you found the API section where creating repos under an organization is explained?
effigies commentedon Aug 27, 2021
It's
POST /org/:org/repos. Need to play with whether one of these orgs should be the name of the org or if it should be part of the repo name in the post data.effigies commentedon Aug 27, 2021
See https://github.com/gogs/docs-api/tree/master/Repositories#create that Yarik posted.
oesteban commentedon Aug 27, 2021
I get unauthorized with the above. The token corresponds to
nipreps-admin, who is an admin of the org.Something obvious I'm not seeing ? (thanks much for the help!)
oesteban commentedon Aug 27, 2021
but now with the right token, gives me
which is confusing and worrisome (that URL pointing at gogs)
EDIT (and SOLVED) - scratch the above, this worked out:
@yarikoptic - the question of how to fix the remote name when the repo is checked out is still standing :(
yarikoptic commentedon Aug 27, 2021
FWIW It is GIN days seems to me. @mih is also working on
create-sibling-gogsdatalad/datalad#5935IMHO it is ok, since gin is just a fork of gogs, and I guess all of the API end points are coming from gogs
if to fix within what git-annex knows about that remote, one way (using the case of recent similar question -- lnnrtwttkhn/highspeed-analysis#2 (comment))
git remote remove gin(to remove it from.git/config)git annex enableremote gin location=https://gin.g-node.org/lnnrtwttkhn/highspeed-decoding- which would fix upremote.logentry for it with the newlocationwithingit-annexbranch and add that remote back to known to git.oesteban commentedon Aug 27, 2021
Okay, I think I got it. I had to checkout
git-annexthen run the two commands you wrote down, commit, push and checkoutmasterback.oesteban commentedon Aug 27, 2021
Cool, all datasets are now mirrored at GIN. I'll figure out next how to make it the default mode of distribution.