Category Archives: git and gitlab

Using Git to write papers

I have been using version control to write my papers for a long time now. I started off using CVS then moved to SVN and now have switched to Git. Though SVN was a little simpler at time, I am happy with my move to  Git.

The basic idea is that there is a server in mathematics holds copies of my papers and all of the changes I have made and committed to the repository. When I go to any of my computers, I just tell it to update to the latest version of the paper. I then can edit it local committing changes as I go. These commit points serve as milestone to which I can return too at any point. Say I think I have a cleaver proof which will simplify my main argument. I start changing the paper only to realize after an afternoon of editing that there is an unintended consequence and I which I had not made all of the changes I did. With a simple command line, I can return to the state the paper was in earlier in the morning. Gone are the endless number of “numbered” versions sitting in the directory. I no longer return 6 months latter trying to recall which was the version I submitted.

While his “versioning” is nice, the real advantage for me is that when I have co-authors everyone can edit the paper at the same time. In the beginning, we should each work on different parts of the paper. Git will merge the sections together effortlessly. Once the paper is mature and we are making smaller edits, we can both roam over the whole paper. On the odd chance that we edit the exact same section, Git will alert me to this and give me a chance to merge the two edits by hand.

As an added benefit, all of my papers are back up with extreme redundancy because of git and I never worry about loosing them. Since I update the directory before I start working, git is also a way to synchronize all of my computers.

I use git even with people who don’t. Some of my collaborators don’t like change or “complicated new ideas” as one put it. Git lets me maintain different branches. I create one for each co-author. I send them the version form there branch and tell them to edit the text. I continue editing my branch (without telling them). When they send me back a version, I switch to their branch and commit the version they sent. Then I merge their branch into mine, thus combining our edits.

How do I use it. The mathematics department runs Gitlab to manage Git repositories. It is a nice easy system for adding users to a project and changing who can edit/see a given paper. I have written a number of pages about how I use git in general, how to set up new repositories, and how git is used for the Math Everywhere class in particular.

I mainly use the command line to manipulate git. However I also sometimes use the free software package called Sourcetree. It is particularly nice to track changes. When I want to find an old version before I mad certain chances, this is how I do it.

Update (June 23, 2017): I now use GitKraken as my GUI instead of Sourcetree. I find it more powerful and intuitive. I can also run it on Linux and install it myself without root permission.

Owncloud and Duke Math

For some time now, I have been using owncloud to sync my personal files. It is a lot like a personal dropbox.

Why do I use owncloud ? It is a great way to seamlessly synchronize file across machines. In this respect, it works a lot like dropbox. It has a client which works with linux, os X (mac) and windows. Initially is seems one can sync 50 GB, though I suspect this is a limit is by Andrew. All of the info stays on Duke servers so there are no privacy issues.

I have not explored the file sharing possibilities much. They exist, but seem not as advanced as dropbox. For two people in the department, they work a lot like Dropbox. There is a shared directory which both people can interact seamlessly. If one person is not in the department,  the sharing options are mainly web based. You can share files or directories secured by a password to anyone with a web accessible link. You can’t seem to seamlessly share a directory with someone else outside the department as one can do in dropbox.  For collaborations such as writing papers of code I personally use Git. It has much better version control than Dropbox or owncloud and it is  free (again hosted in the Duke Mathematics department).

There is a web interface at https://cloud.math.duke.edu/owncloud/ . From there one can create links to share a file with someone or retrieve a file when you only have web browser access.

There is also an ipad/iphone client which costs a few dollars. I use it regularly to save file from my phone to my work account. There is also a webdev address

https://cloud.math.duke.edu/owncloud/remote.php/webdav/

 which is useful for some other ipad/iphone clients.

To get started, type

% owncloud

at a linux command prompt on a department machine. It should step you through the needed steps. It will create a directory called “owncloud” in your home directory.  Any file in this part of your directory tree will be synchronized.

Gitlab at Duke Math

I am now using gitlab to manage my git accounts. I used to use gitolite. I liked it very much, but the department wanted others to use git and gitlab is better to allow the entire department  access.

To use gitlab you have to set up our account.  Follow the steps below to set up your account.

  1. you should have gotten an email  from gitlab. If you lost it, you can just follow this link and say that you forgot your password. It will put you through the steps to get a new one mailed to you.
  2. Once loged it go to this page to up load your ssh key. The ssh key is on the driectory ~/.ssh and is usually in a file called
     ~/.ssh/id_rsa.pub
  3.  You are now ready to use gitlab. See this page for a description of using it for math everywhere. And look at this page for a description of commands one typically uses.

Using Git – typical commands

Committing changes locally

After  you have made some changes  to your local copy you might want to commit them so that they are saved and you can “revert” back to the current state should you ever want to. Each “commit” amounts to a snap shot of the directories which  is saved for all time. You can always revert to the current state.

% git commit -a -m "some note saying what you did"

The “-a” tells to commit all changed files. This is my usual practice. The “-m” adds a memo describing what you changed. If you leave off the “-m” and the comment which follows the system will open an editor (if your system is set up correctly) and you can edit the comments in the editor.

Pushing your changes to the master repository

Once you have committed your changes locally, you have to “push” them to the main repository. This is done with

% git push

If there is an error it is most likely because changes have been made to the main repository since you last updated. You need to perform a “pull” to update your local copy of the repository. You will then often be required to commit again to add the effects of “merging” your local repository   with the main one. Hence a typical “workflow” for committing onese changes is :

% git pull
% git commit -a -m "merges"
% git push

Updating the local repository

This is done by pulling the changes.  Before you can pull you have to commit any changes you have made locally. So although to pull one only needs to say

% git pull

Operationally one usually runs:

% git commit -a -m "my changes"
% git pull

 

Adding a file to the repository

git only tracts the files you tell it to. To add a new file called “new-file.tex” to version control tracking you would type

% git add  new-file.tex
% git commit -a -m "added the file new-file"

Removing a file from the repository

To remove a file, one types

% git rm  new-file.tex
% git commit -a -m "removed the file new-file"

Of course the file will still be in previous versions and can be recovered at any moment. It will just no longer be in the current version.

Renaming or moving a file in the repository

Renaming is the same as moving.

% git mv  new-file.tex different-name.tex
% git commit -a -m "changed name of new-file"

 

Git – Setting up new repositories

Setting up a new repository – locally

Here is the standard way to start a new repository from an existing directory. The first part just tells creates some content. You really just need to start from the “git init” line and add all the files you want with “git add”.

mkdir api-test
cd api-test
git init
touch README
git add README
git commit -m ‘first commit’

 

Pushing to a duke math gitlab repository

Assuming  I have already set up a get lab repository called api-test the following command pushes the git repository I set up above to that repository. Of course you would replace “jonm” with your username. gitlab actually tells you what to so.

git remote add origin git@git.math.duke.edu:jonm/api-test.git
git push -u origin master

Math Everywhere and Git

Git is a version control software which will allow use to keep the files up to date and prevent the proliferation of different version. It has a number of features

  • We can never “loose” a file. We can always roll things back to a previous date. (This assumes that changes are regularly checked it as described below.)
  • One can edit the files on you laptop and then merge them in to the online site when one is ready for them to be seen.
  • We can all work at once with out much danger of having conflicting edits.
  • If we want to start an experimental branch we can. Or a branch for a future instructor to edit with out letting them change the main branch.

The basic model is that each person has there own copy of the repository. They work on it locally checking in their changes  to the local repository. Then when they are ready they “push” their changes to the main repository. At any time they can update their local repository by “pulling” the changes others have made. You can have as many copies of the repository as you want. That way you can have one on your laptop, work and home machine.

Running “ssh-add” as described here will keep you from having to repeatedly enter your ssh password.

The first time you run git

The very first time you run git on a machine you need to run the commands below with “John Doe” replaced with your name and the email replaced with yours. This lets git know who you are.

% git config --global user.name "John Doe"
% git config --global user.email johndoe@example.com

You should only have to do this once on each machine.

Making a local Copy of Matheverywhere

To make a local copy you “clone” the master repository. Changing to the location where you want the local directory copy to be, one types (all on one line):

% git clone git@git.math.duke.edu:jonm/math-everywhere.git matheverywhere

This will create a copy of the math everywhere repository in a directory “matheverywhere”. To have it called something else replace the last “matheverywhere” in the above command with some other directory name.

Now what ?

Now you have a local copy of math everywhre. You can edit this all you want and not change anything on line until you are ready.

  1. When you are ready you should “commit” the files to you local repository.  (see here for directions)
  2. “push” the file to the central  repository (see here for directions)
  3. Log in to one of the math server and “pull” the update to the web server. The directions for this are given below.

 

Updating the version on the web server

The version that the web server uses is just a repository like any other repository.  You need to “pull” the latest version to update it.

On a math department machine:

  1.  change to the math_everywhere directory
    % cd /home/system/httpd/htdocs/courses/math_everywhere
  2. pull the latest version
    % git pull
  3. make sure that the permissions  have been set correctly
    % chmod -R ag+rX .

If you find you keep having to type in the password for your ssh key repeatedly. You should use the ssh-add command to make it so you only have to type it once per session.

% ssh-add

If you have ‘sshed’ into the math machine, you should follow the directions here to use ssh-agent.