Perforce as the version control system at Google
It is well known that Google uses Perforce as its internal source management system (it has a source license). Niall Kennedy writes:
Google uses a company-wide Perforce depot with almost no developer branches. Each developer has their own NFS workspace readable by anyone in the company, including automated processes. An administrative process takes snapshots of each developer workspace including local development environments accessed over SSH. Files within these snapshots can be compared to checked-in data, encrypted, and archived.Dan Bloch did a presentation at Perforce European User Conference, called “Performance and Database Locking at Large Perforce Sites”. It contains statistics on Google Perforce Depot, such as:
(via Niall Kennedy: “Google Mondrian: web-based code review and storage”)
- More than 3000 users and 100Gb of metadata on one primary server;
- Hardware is an HP DL585 4-way Opteron with 128Gb of memory;
- Depot is on a NetApp filer;
- Metadata and journal on RAID-10 local disk;
Recent posts on similar topics
- Robin Luckey: "The World's Oldest Source Code Repositories" - October 18th, 2007
- Tim O'Reilly: Why Congress Needs a Version Control System - October 7th, 2007
- Linus Torvalds: Re: clarification on git, central repositories and commit access lists - August 27th, 2007
- Upgrading Drupal with Git - August 2nd, 2007
- Mozilla: Version Control System Shootout Redux Redux - June 24th, 2007
December 10th, 2006 at 6:27 pm
I liked learning about the way they use their version control system.
But a couple of questions showed up while reading:
- While don’t they use several branches? Well, my oppinon can be a bit biased, but Perforce is not very good on that… is maybe the reason?
- Why do they export their workspaces using NFS?? Why don’t they use the SCM for that?? I mean, they could use “branch per task” or “branch per developer” and have it inmediately solved…
What do you think?
pablo
December 10th, 2006 at 11:18 pm
hi,
I believe it is because Google mostly does in-house development. The quote says “almost no developer branches”. I guess that things that they do release externally, such as Google Talk and Google Desktop, have their (maybe short) branches with fixes etc.
All other things I think just have rather short release cycle, so why bother.
As for exporting using NFS, I think this just means that every developer can take a look into another developer’s $HOME, so that’s more of an open attitude.
“and have it immediately solved” — what is “it”?
Thanks,
December 11th, 2006 at 10:37 am
Well, what I wanted to say was that branches are useful not only to split development, but also to coordinate teams.
In the case you’re describing, suppose every developer has its own branch (branch per developer pattern). Ok, if developer B wants to see what developer A is doing (to have a look, to help, whatever), he just takes his branch. So, no need to access his home by NFS, just use the SCM.
We internally use “branch per task” for our own development, so switching tasks is quite easy, and also sharing our work just “pointing” to what other developer is doing…
pablo
December 12th, 2006 at 1:18 am
pablo,
I’ve never heard of “branch per developer” pattern, and I cannot really understand its usefulness.
I’ve found several interesting links to blog while researching the issue, thanks!
Also, I’ll take a look at what PlasticSCM does.
March 10th, 2007 at 11:14 am
[…] I knew Google was using Perforce for awhile. Today I ran into this blog entry and what piqued my interest was the setup and the way Google uses Perforce. Check out that page for details. I also highly recommend checking out the presentation slide on Performance and Database Locking at Large Perforce Sites. Moreover, Google has a custom code rewiew application called Mondrian, written in Django. Mondrian is a web-based code review system built on top of a Perforce and BigTable backend with a Python-powered front-end. […]