Just a short one today.

Say you have a repository that contains a submodule. But, for whatever reason, you want to change the URL that the submodule points to. This could be because that repo moved, or changed it’s name. Or maybe you’ve forked that repository and you want to use your fork as a submodule.

All you need to do is to change the URL in the “.gitmodules” file (I wrote about doing this in my last post). Now, all you need to do is to run the command from the top level of the repository:

git submodule sync

This will update the local repository settings in “.git/config”.

Note

If you have more than one submodule, this will update all the submodules in “.git/config” the the URLs in “.gitmodules”. If you’ve set up your local repository with custom URLs (again, which I wrote about last time), this might not be what you want. You can choose to update submodule explicitly with:

git submodule sync -- name

which will only sync the repository called “name”, which you can get from the “.gitmodule” file in the “Submodule” section. Be mindful of the spaces before and after --.

Say you want to add a submodule to your repository, and this submodule is a one to which you have push access. It is frequently useful to be able to publish a repository with the submodule URL as pull-only, while keeping the push-pull URL in your own repository. This way, you can make changes to the submodule repository from within the enclosing repository. This is a simple thing to do in Git once you know how submodules are arranged.

Example

For convenience, let’s use a a couple of GitHub repositories as an example. Let’s start with the Hello repository, and say I want to add the Just Testing repository as a submodule. That’s simple enough; I check out the Hello repository to my local drive:

cd tmp
git clone -o github git@github.com:Abizern/Hello.git
cd Hello

(You may notice the unfamiliar -o github in the clone command. This names the remote for this clone “github” instead of the default “origin”. Git is distributed, and I often have more than one remote for repositories, so I like to name my remotes semantically. Another name I use is “dropbox” for obvious reasons).

Now I’m in the repository I add the remote. This is where I have to make a choice. GitHub gives you a choice of clone URLs:

If I use the Read+Write URLs, for the submodules, and someone else clones my repository, they will be using URLs to which they do not have access. So, cloning the repository will work, but when they go to initialise the submodule, the command will fail because they can’t access the repository through that URL. But, If I use the Read-Only URL, I won’t be able to push to my submodule repository. That’s okay, because I can add the repository with Read+Write access for me but change it for the shared repository. Let’s start by adding the submodule to the repository using the Read+Write URL:

git submodule add git@github.com:Abizern/JustTesting.git

This will add a directory called “JustTesting” to my repository and check out the files. You can check the submodule with git submodule status which, in my case, shows this:

 a73686133fc66884faac1f8c1d9603b9ffcf09b1 JustTesting (heads/master)

Which shows the sha of the checked out commit of the submodule (there are no - or + adornments so there are no issues) the directory which contains it (JustTesting) and the current description (in this case it just shows the head of the master branch. If there were any tags in the repository it would show the output of git describe).

Checking the status of the repository with git status shows that we have two new files:

JustTesting isn’t a new file, it’s just the way that Git sees a submodule. What we are interested in is the “.gitmodules” file. You can see that this shows where the the submodule is kept and what it’s URL is. Now let’s look at the way the submodule is configured:

First I find out what the name of the remote is (yes, I know it’s origin) and then I get information about the remote. This shows that it is configured to the Read+Write URL

Let’s take a look at the “.git/config” file. This is where the configuration for the local repository is kept:

The interesting section is at the bottom, which has information about the submodule. This looks a lot like the entry in the “.gitmodules” file, except that it doesn’t reference the path to the submodule.

Now let’s change the URL in the “.gitmodules” file so that it points to the Read-only URL.

So there are two URLs for this submodule. The repository itself contains the Read-Only URL in “.gitmodules” and this is the configuration that is shared among all repositories that clone it. And the private Read+Write module is configured in the “.git/config” file which applies only to this particular repository. Now I just need to commit the changes and I can push this back up to GitHub.

Go and look at the Hello repository for yourself. You can see that JustTesting is a submodule. And if you clone this and update the submodule, it will be configured to point to the Read-Only version of JustTesting:

cd tmp
git clone -o github git@github.com:Abizern/Hello.git Hello2
cd Hello2
git submodule update --init

And you can see for yourself that “.gitmodules”, “.git/config” are configured for the Read-only URL. I’m not going to show you the output of those because I want to show you something more interesting.

Imagine I am at my other machine and I want to clone this repository but still have Read+Write Access to the submodule. This is how I go about that. I clone the repository as normal but I don’t update the submodule in the same way as I just did above

cd tmp
git clone -o github git@github.com:Abizern/Hello.git Hello3
cd Hello3

Then – I init the submodule:

git submodule init

And you can see that all this does is register the submodule, but does not actually check out the repository:

Now - I can change my local configuration in “.git/config” so that the submodule URL is the Read+Write one:

And now I update the submodule repository:

git submodule update

And I have Read+Write access from my local submodule:

Explanation

That seems like a lot of steps - but only because I went through it in detail. This is why it works:

URLs for submodules are kept in the “.gitmodules” file. As this is in the top level of the repository, it is shared amongst all repositories that clone from it. When you initially add a submodule to your repository, an entry is added to the “.gitmodules” file and to the local “.git/config” file and the repository is checked out. All in one step (although the changes to the repo have to be committed manually, this allows you to change the public facing configuration).

When you clone a repository that has submodules, all you are getting is the “.gitmodules” file. It also gets an entry that refers to the sha of the submodule to check out, I’ve written about this previously. It takes two steps to actually set up the repository. First, git submodule init copies the submodule settings from “.gitmodules” to “.git/config” and then git submodule update pulls down the repository at the sha that is referenced. Normally, you can do this with one step: git submodule update --init as in the Hello2 example. But If you do it in two steps, you have a chance to edit the “.git/config” file before the repository is pulled down, which means you can customise the way that the submodule is set up in your own repository.

There is a flag to the git merge command: --squash which at first looks as if it does the same thing as a rebase, but this isn’t exactly so. This post attempts to explain the difference.

Consider the following repository structure:

There is a master branch and two feature branches feat1 and feat2. The feat2 branch will introduce a merge conflict, which is why it is highlighted in red.

Say you want to bring the two features into the master branch as a single change. There are, as you would expect with Git, multiple ways of doing this.

Merging

One way to do that is by merging the two branches into the master. From the master branch you can just git merge feat1 feat2 (yes, you can merge multiple branches in one step) You will get an error because of the merge conflict, which you should know how to fix by now, and this will give you this structure:

Rebasing

That’s fine if you want to be able to show that the merges have happened, but sometimes, that isn’t the case and you want to be able to show a clean history and a single change. To do that, you can use the rebase command. First by rebasing feat1 onto feat2 git checkout feat1; git rebase feat2

And then you can rebase this branch onto the master with git rebase --interactive, where you can squash the three individual commits to a single commit, and resolve the merge conflict introduced by feat2:

And now, switch to the master branch and rebase it onto the feat1 branch (or merge it, the result will be the same as it will resolve as a fast-forward merge). This will give you a single commit with a clean history and all the merged changes from feat1 and feat2:

git merge —squash

Now to the topic of this post; using the --squash flag with git merge. What this does is similar to an interactive rebase where you squash all of the commits, except it doesn’t check in the change, it just updates the index (and by extension, the working directory). So, starting from the original repository structure we can get the same result by first git merge --squash feat1 feat2 and fixing the merge conflict. This won’t affect the the checked in state of the master branch, but your index will have all the changes:

What’s the advantage of this? Well, you can get a clean history by just commiting the changes:

But, even more interestingly, you can now back out the changes from the index (which leaves them in the working directory) and add back selective changes using git add --patch and create an entirely different set of commits.

Which should you use?

For most cases, merging would be the choice of most people. It’s simple enough, and if there aren’t many micro commits the history will be coherent. In this example it only takes three steps (1. merge 2. fix conflicts 3. commit).

If you are the sort that makes many micro commits (like me) and you then rebase them into larger chunks before merging the changes to another branch, then the rebasing method is the one you are already using, even if you squash all the changes into a single commit. I prefer this method, because you have greater flexibility with what changes you make. I also find it easier to handle merge conflicts this way. But it does take longer (1. rebase feat1 onto feat2 2. Interactive rebase 3. Amend the commit messages 4. fix conflicts 5. commit 6. rebase master onto feat1).

But, if you’re aware of what conflicts might occur and how to resolve them, or if you want to use interactive patching, then using git merge --squash is a good choice; and it might be faster (1. merge 2. fix conflicts 3a. commit or 3b/3c … patch add the changes)

If you have a repository with many submodules, it’s easy to update them all at once.

For example: I have my vim dotfiles in a Git repository. Not only does this make it easier for me to keep my vim setup the same across multiple machines, but also lets me add other bundles as submodules which makes it easy to update the bundles and propagate changes.

Once in a while I go into the repository and run:

git submodule foreach 'git pull || :'

The foreach command evaluates a shell command in each of the submodules. In this case git pull. The || : (in bash this is ‘or true’) stops the foreach command from terminating because of a non-zero return from the shell command.

Or, you could avoid the zero return problem by echoing the return value:

git submodule foreach 'echo `git pull`'

This is just one use of foreach. Since I don’t actually develop the submodules, I can keep the repository tidy by running:

git submodule foreach 'echo `git gc --aggressive`'

I’ve previously written about how to get the diffs between your working tree, the index and the last commit. But you frequently need to see the difference between the code you’re working on and previous commits.

Here’s a diagram that summarises the commands:

If you’ve read the previous article then this should make sense I’ve used HEAD~ to represent the parent of the tip of the branch you’re working on, but this can be replaced with the any of the other methods of referring to commits specified in the gitrevisions documentation.

However, when doing this, you usually only want to see changes between specific files. For this you can optionally pass in the paths that you want the diff between after a pair of dashes --. If you leave out the paths you’ll get the complete diff.

1. git diff --cached HEAD~ -- somefile.py
   This will show the difference between the version of somefile.py in the index and the version in HEAD~

2. git diff HEAD~ -- somefile.py
   This will show the difference between the version of somefile.py in the working tree and the version in HEAD~.

If you are using an external viewer such as Changes or Kaleidoscope (for the Mac) or other tools for other platforms. These usually come with instructions for setting them up and calling them using the git difftool command rather than git diff. The good news is that the options and parameters are the same so you can still get pretty diffs with the same control.

Unsurprisingly, there is a convention for writing Git commit messages. This comes from the submitting patches guidelines for Git itself. In summary:

• The first line of the commit message should be a short description (50 characters is the soft limit), and should skip the full stop
• The body should provide a meaningful commit message, which:
  • uses the imperative, present tense: “change” not “changed” or “changes”.
  • includes motivation for the change, and contrasts its implementation with previous behaviour.

Most tools only show the first line of the commit message, which is usually enough to know what it is about. It’s like the subject line in an email: you don’t need to see the whole message to know whether you are interested in it. The 50 character soft limit is to play nicely with command line displays. If you use vim as a commit message editor it knows about this soft limit, which is why the colour changes after the 50th character. Personally, I go past this if I have to (which is most of the time because of the long names that are used in Cocoa).

The use of the imperative, present tense is one that takes a little getting used to. When I started mentioning it, it was met with resistance. Usually along the lines of “The commit message records what I have done”. But, Git is a distributed version control system where there are potentially many places to get changes from. Rather than writing messages that say what you’ve done; consider these messages as the instructions for what applying the commit will do. Rather than having a commit with the title:

Renamed the iVars and removed the common prefix.

Have one like this:

Rename the iVars to remove the common prefix.

Which tells someone what applying the commit will do, rather than what you did. Also, if you look at your repository history you will see that the Git generated messages are written in this tense as well - “Merge” not “Merged”, “Rebase” not “Rebased” so writing in the same tense keeps things consistent. It feels strange at first but it does make sense (testimonials available upon application) and eventually becomes natural.

Having said all that - it’s your code, your repository: so set up your own guidelines and stick to them.

If, however, you do decide to go this way then git rebase -i with the reword option would be a good thing to look into.

After my post about signed tags yesterday Kevin Ballard left a comment about a neat trick that lets you put the gpg public key used to sign tags into the repository itself. This explains how to do just that.

As a practical example I’ll use this Hello World repository on GitHub.

What this technique does is to add the gpg public key as a blob to the repository, but not as part of any branch. Normally, this would be cleaned up by git’s garbage collection, but by tagging this blob it gets a reference which means it isn’t orphaned so it persists.

So the repository will look like this:

It has one branch master and two tags v1.0 that tags version 1 and pubkey that tags the public key.

The project was created and checked in as normal. But how do you add a file by itself without adding it to the project? We do this by adding it as a raw blob to the repository.

Create the tagged public key

First. Lets get the gpg public key that we need. The email I am using for this repository as abizern@abizern.org (and I’ve set this up in the config file) so to get the public key as a plain text file I run:

gpg --armor --export <abizern@abizern.org> > pubkey.txt

The actual name of the file doesn’t matter because a blob only stores the contents of a file. Now we can turn this file into a blob with:

git hash-object -w pubkey.txt

This will compute the hash of the object and the -w flag gets the object written into the object database. It returns the hash value. you want to copy this. In my case it was

ab147894b92328ffef9db011789e4d0e55652688

So now I can tag this blob, even though it isn’t in the index or part of the tree of commits:

git tag -s pubkey ab147894b92328ffef9db011789e4d0e55652688

This creates a signed tag called pubkey that points to the blob that contains the public key. I didn’t need to use a signed tag, but it doesn’t hurt.

Now, I don’t need the pubkey.txt file so I can delete it. Running git status now should show that there is nothing more to do.

You can now tag other commits, as I have done with v1.0, and sign them. Remember when pushing your changes to remote repositories you need to push tags explicitly. You can use git push origin --tags to push all tags or git push origin pubkey to push just the pubkey tag (assuming your remote repository is named origin).

Verify signed tags.

Now that the public key is in the repository, anybody who clones the repository can add the public key to their gpg keychain by:

git cat-file blob pubkey | gpg --import

This gets the contents of the blob pointed to by pubkey which is the text version of the public key formatted for export and imports it to your own gpg keychain.

Now they can verify a signed tag, for example v1.0 by:

git tag -v v1.0

and the output will be something like this:

object d56fe222cd3d5fe5ac1709368c5c09cd496932df
type commit
tag v1.0
tagger Abizer Nasir <abizern@abizern.org> 1295369217 +0000

Tagging version 1.0
gpg: Signature made Tue 18 Jan 16:46:57 2011 GMT using RSA key ID 65875E7F
gpg: Good signature from "Abizer Nasir <abizern@abizern.org>"
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: 9CC3 9E69 683B DD90 36DD  4622 FFD4 A90C 2D9E EF7B
Subkey fingerprint: 83B8 FAA4 62BD 4821 49B3  5FE3 4F65 DD8F 6587 5E7F

The important thing here is to see the line that says that this is a good signature. The warning just says that you only have my word that this signature actually belongs to me. Those warnings tie in to the GPG web of trust which is beyond the scope of this site.

Have a look at the example repository and try it for yourself. As an exercise for the reader, why not sign my public key in your keychain so you don’t get these warning messages.

FAQ

Why can’t I see the pubkey tag in my Git GUI?

Some of the visualisation tools, such as gitk don’t show tags that point to objects, only tags that point to commits. Which is why it is useful to use the command git tag to show the tags in a repo.

How do I see what’s in a tag?

you can use git cat-file -p <tagname> or git show <tagname> which also shows a diff.

How do I get the SHA1 of the tag?

If you use the git show-ref --tags command it brings up the SHA1 and the tag name. This is where you can get the SHA1 of the tag that points to the public key and post it somewhere so people can verify that the tag (and hence the public key) has not been changed.

I’ve already written about the Git Tag Object and made a passing reference to signed tags.

Git doesn’t make any checks about whether the name and email you enter into a git config file really is you, so it’s trivial make a check-in that looks like it’s been made by someone else. But, if you have some form of GPG installed, you can cryptographically sign a tag object (not a lightweight tag) than can be verified as being created by you.

Since a tag object references a commit, which in turn references another commit, and so on, a tag verifies a whole tree of commits. Changing any of the commits, or the tag, would mean new objects would be created that would break this tree. So by signing a tag, which points to a commit, a whole section of the code history is being validated by someone. And if you trust that person, and you trust his public key, you can verify that the tag was created by that person.

To create a signed tag, just pass the -s flag instead of the -a flag:

git tag -s -m"tagging version 1.0" v1.0

This will create a signed tag object called v1.0 with the passed commit message and it will use the signing identity of the committer (if there is a key for that). The documentation for git-config states that you can set a key for user.signingkey that will be used by default to sign commits, but I’ve found that I don’t need to do this as the correct key is used based on the user.email that I am using. A prompt will come up asking you for your key’s passphrase.

If you want to use a particular signing identity when tagging then use then use:

git tag -u <key-id> -m"tagging version 1.0" v1.0

Here, the -u <key-id> specifies the key to use.

Also, as with commit messages, if you leave out the message an editor will be brought up for you to enter the tag message.

The signed tag object is only slightly different from the unsigned tag object in that it has a PGP signature block. To verify the signature use the -v flag of the tag command

git tag -v v1.0

and if you have the correct public key for the user in your GPG Keychain it will let you know if the signature is valid or not.

In my earlier post I also mentioned that a tag object does not have to point at a commit; it can point to any other git object. In this way, you can also tag individual files or trees or even other tags if you are paranoid enough to want to. But this does nothing more than putting a name to a tag, or a commit. It establishes nothing more than responsibility - not a guarantee of quality nor of the origin of the work.

On Mac OS a and Unix systems Git defaults to using vim as the editor for commits.(I don’t know what the default is on Windows) You can change this, if you like, but for now I’ll just give a couple of tips for using vim — just enough to get by.

When Git needs a message for an action it will bring up vim in view mode. You can move around using the h(move left), l(move right), k(line up) and j(line down)

But for quick editing, the i key will put you in editing mode where you can type and move around as you would for most editors. When you’ve finished press the esc key to get you out of editing mode.

The normal command to save the message and quit is :wq which is bit of a finger twister. An alternative is ZZ which does the same thing, but it’s a much easier key combination to use.

A final thing. An empty commit message cancels the commit, (or rebase or tag, or whatever action the message is being added for). So if you change your mind just delete the message (the lines starting with the comment character # don’t count). A useful way of doing this is to make sure you’re not in insert mode by pressing the esc key and then using dd which deletes the whole line.

I use Git. I use Mercurial, sometimes, but I use Git. I used to find it hard going (actually, I still find it hard going, but it’s a lot easier now). I used to find it frustrating that you pull from origin/master, but push to origin master; and how the documentation talks of things like refspecs, HEAD, and ORIGIN_HEAD.

Really; you don’t want to wade through stuff like that when you’re trying to fix a merge, or push to a colleague’s repository. You just want to get stuff done. I imagine Matt Gemmell pulling his hair out when using it.

A while ago I read a piece by Neal Stephenson where he described Unix as the “Hole Hawg” of operating systems. (It’s a short piece, go read it). I think of Git as the Unix, and Mercurial as the Mac of version control. It’s complicated and can really mess you up but there is a lot of power there and for the times that you need it; you’ll be glad of it. Most of the time a quick Google search is enough to answer questions.

There is more than one way of using Git. If you look at some of the comments on here you’ll see that my approach isn’t necessarily shared. I welcome these comments. I’ve learned from them and I’m grateful that someone took the time to share their views. If I don’t reply it isn’t out of churlishness; rather that I have nothing to add. I’m sure that there will be many more over the year.

My intention with this site is to set up a resource of quick answers along with longer explanatory posts and add to the list of Googleable resources. It’s also a way of pushing myself to learn more about Git as I write them. It isn’t to push a “One True Way”. But with a little thought and understanding using Git isn’t that difficult. Yes, it can be dangerous to rewrite history. Yes you can lose your work if you delete a branch. Yes you can mess up XML files with a merge. But these risks can be minimised with a few simple rules. If you can figure out the memory management rules in Cocoa, you can work with Git (and even if you can’t you can). And with a little knowledge and confidence you can develop your own, consistent, Git workflow.

But, for the love of whatever you hold holy; don’t just use it as you would Subversion