August 22, New Views on your History with git replace. Christian Couder

Transcription

1 August 22, 2014 New Views on your History with git replace Christian Couder

2 About Git A Distributed Version Control System (DVCS): created by Linus Torvalds maintained by Junio Hamano since 2005 preferred VCS among free and open source developers

3 About myself started developing Git in 2006 worked on git bisect developed git replace in 2009 $dayjob at SoftAtHome

4 Git Design Git is made of these things: Objects Refs config, indexes, logs, hooks, grafts, packs,... Only Objects and Refs are transferred from one repository to another.

5 Git Objects Blob: content of a file Tree: content of a directory Commit: state of the whole source code Tag: stamp on an object

6 Git Objects Storage Git Objects are stored in a content addressable database. The key to retrieve each Object is the SHA-1 of the Object s content. A SHA-1 is a 160-bit / 40-hex / 20-byte hash value which is considered unique.

7 Blob SHA1: e blob <size> blob = content of a file /* content of this blob, it can be anything like an image, a video,... but most of the time it is source code like:*/ #include <stdio.h> int main(void) { printf("hello world!\n"); return 0; }

8 Example of storing and retrieving a blob $ echo "Whatever..." git hash-object -w --stdin aa eea6d8e0bc68f3663de51767a9f5b1 $ git cat-file -p aa Whatever...

9 Tree SHA1: 0de24... tree = content of a directory tree <size> blob hello.c e It can point to blobs and other tree lib 10af9... trees.

10 Example of storing and retrieving a tree $ BLOB=aa eea6d8e0bc68f3663de51767a9f5b1 $ (printf " whatever.txt\0"; echo $BLOB xxd -r -p) git hash-object -t tree -w --stdin 0625da548ef0a7038c44b480f10d5550b2f2f962 $ git cat-file -p 0625da548e blob aa whatever.txt

11 Commit SHA1: 98ca9... tree commit 0de24... <size> commit = information about some changes parents author committer () Christian <timestamp> Christian <timestamp> It points to one tree and 0 or more parents. My commit message

12 Example of storing and retrieving a commit (1) $ TREE=0625da548ef0a7038c44b480f10d5550b2f2f962 $ ME= Christian Couder <chriscool@tuxfamily.org> $ DATE=$(date "+%s %z") $ (echo -e "tree $TREE\nauthor $ME $DATE"; echo -e "committer $ME $DATE\n\nfirst commit") git hash-object -t commit -w --stdin 37449e a0a888ee100cfd0a7ba7927b3

13 Example of storing and retrieving a commit (2) $ git cat-file -p 37449e9554 tree 0625da548ef0a7038c44b480f10d5550b2f2f962 author Christian Couder <chriscool@tuxfamily.org> committer Christian Couder <chriscool@tuxfamily.org> first commit

14 Git Objects Relations SHA1: e84c7... commit <size> tree 29c43... parents () author Christian <ts> committer Christian <ts> Initial commit SHA1: 29c43... tree <size> blob hello.c tree doc 0de ca9... SHA1: 0de24... blob <size> int main() {... } SHA1: 98ca9... tree <size> blob readme 677f4... SHA1: 98ca9... blob install 23ae9... commit <size> tree 5c11f... parents (e84c7...) author Arnaud <ts> committer Arnaud <ts> Change hello.c SHA1: 5c11f... tree <size> blob hello.c bc tree doc 98ca9... SHA1: bc blob <size> int main(void) {... }

15 Git Refs Head: branch.git/refs/heads/ Tag: lightweight tag.git/refs/tags/ Remote: distant repository.git/refs/remotes/ Note: note attached to an object.git/refs/notes/ Replace: replacement of an object.git/refs/replace/

16 Example of storing and retrieving a branch $ git update-ref refs/heads/master 37449e9554 $ git rev-parse master 37449e a0a888ee100cfd0a7ba7927b3 $ git reset --hard master HEAD is now at 37449e9 first commit $ cat whatever.txt Whatever...

17 Result from previous examples master commit 37449e9554 tree 0625da548e blob aa

18 Commits in Git form a DAG (Directed Acyclic Graph) history direction is from left to right new commits point to their parents

19 git bisect B B introduces a bad behavior called "bug" or "regression" red commits are called "bad" blue commits are called "good"

20 Problem when bisecting Sometimes the commit that introduced a bug will be in an untestable area of the graph. For example: W X X1 X2 X3 Y Z Commit X introduced a breakage, later fixed by commit Y.

21 Possible solutions Possible solutions to bisect anyway: apply a patch before testing and remove it afterwards (can be done using "git cherry-pick"), or create a fixed up branch (can be done with "git rebase -i"), for example: X + Y X1' X2' X3' Z' W X X1 X2 X3 Y Z Z1

22 A good solution The idea is that we will replace Z with Z' so that we bisect from the beginning using the fixed up branch. X + Y X1' X2' X3' Z' Z1 W X X1 X2 X3 Y Z $ git replace Z Z'

23 Grafts Created mostly for projects like linux kernel with old repositories..git/info/grafts file each line describe parents of a commit <commit> <parent> [<parent>]* this overrides the content in the commit

24 Problem with Grafts They are neither objects nor refs, so they cannot be easily transferred. We need something that is either: an object, or a ref.

25 Solution, part 1: replace ref It is a ref in.git/refs/replace/ Its name is the SHA-1 of the object that should be replaced. It contains, so it points to, the SHA-1 of the replacement object.

26 Solution, part 2: git replace git replace [ -f ] <object> <replacement> to create a replace ref git replace -d <object> to delete a replace ref git replace [--format=<format>] [ -l [ pattern ] ] to list some replace refs

27 Replace ref transfer as with heads, tags, notes, remotes except that there are no shortcuts and you must be explicit refspec: refs/replace/*:refs/replace/* refspec can be configured (in.git/config), or used on the command line (after git push/fetch <remote>)

28 Creating replacement objects When it is needed the following commands can help: git rebase [ -i ] git cherry-pick git hash-object git filter-branch (or maybe tools like BFG Repo-Cleaner)

29 What can it be used for? Create new views of your history. Right now only 2 views are possible: the view with all the replace refs enabled, or the view with all the replace refs disabled, using --no-replace-objects or the GIT_NO_REPLACE_OBJECTS environment variable

30 Why new views? split old and new history or merge them fix bugs to bisect on a clean history fix mistakes in author, committer, timestamps remove big files to have something lighter to use, when you don t need them prepare a repo cleanup mask/unmask some steps (see git imerge)...

31 Limitations everything is still in the repo so the repo is still big there are probably bugs insecure? complex? confusing?

32 Current and future work git replace --edit git replace --graft fix bugs allow subdirectories in.git/refs/replace/ maybe allow views as set of active subdirectories...

33 Conclusion best of both world: immutability and configurability of history no true view history is important for freedom

34 Many thanks to: Junio Hamano (comments, help, discussions, reviews, improvements), Ingo Molnar, Linus Torvalds, many other great people in the Git and Linux communities, especially: Andreas Ericsson, Johannes Schindelin, H. Peter Anvin, Daniel Barkalow, Bill Lear, John Hawley,... LinuxTag organizers and attendants, SoftAtHome, the company I am working for.

35 Questions?

36