Mostly written 2002-2003. Please report errors to bradb@shore.net
The codeline is the foundation for Perforce.
In Perforce, codelines are simply naming conventions. Once set up, these conventions are difficult or impossible to change. So it is worth some thought ahead of time. The goal is to categorize all source files into codelines, while keeping the number of file revisions low. From trial-and-error, the following organization should work well.
To create the xyz depot
p4 depot xyzp4
depotsp4 protect -- add entries for //xyz/...If you get an error like "Mapping '//customer/...' is not under '//road-disk/...'", Try creating the default client first and then edit it.
Perforce stores text files as RCS ",v" files in
/work/perforce/depots
Perforce stores binary files as compressed files (".tgz") in ",d" directories. All files are owed by 'perforce' with group 'perforce'. Under Windows, all files and directories are lower-case.
Projects and branches should be controlled by 'p4 protect'. Otherwise, a misspelling can permanently create a new project or branch. On Windows, incorrect capitalization can have the same effect.
To create the xyz project or branch
p4 protect -- add entries for xyz/...
See
p4 changes -m 1; cp $p4p4/db.user $p4p4/db.group $p4p4/db.protect $p4p4/p4d.* $p4p4/README . ; p4 changes -m 1
scp * bug-ex:/work/perforce
To install the license file on each server, do the following:
mv license license.9.10.01sudo /etc/init.d/p4d stop
sudo /etc/init.d/p4d startPerforce upgrades its server and clients twice a year. New clients can run with old servers. To upgrade the clients and user documentation
The Perforce documentation has detailed instructions for upgrading a server. Here are the steps we use at ATG [rjohnson, Michael Shields, bbarber]:
Periodic deleting of the abandoned clients is certainly something that should be suggested for a site such as yours. In general, deleting abandoned clients will reduce the size of your db.have table, which can make for more efficient access.
super user bbarber * //...
p4 verify -qu //... >& verify.log
Note: the verify run will take 4-5 hours on geezer //... [5/02]
harry-lime verify
/etc/init.d/p4d stopsu perforce
/work/perforce/p4d.crondailycd /work; /road/local/bin/tar -zcf perforce_bak.tgz
perforcemv /work/perforce/p4d /work/perforce/p4d.old
cp /home/rjohnson/newperforce/p4d /work/perforce
chmod 755 /work/perforce/p4dcd /work/perforce
./p4d -xusu root
/etc/init.d/p4d startp4 verify -q //...Remove your super user access.
NOTE: the upgrade will require about three times as much disk space as all of the current db.* files added together. Once the upgrade is finished, the db.* files should be a bit smaller. The total upgrade time for geezer is around 12 hours.
To delete a user, first delete the user's clients.
To reinstate a user, recreate the user with the same userid. Perforce uses the userid to indicate the user (e.g., change lists).
To delete a user:
These files may contain valuable edits on the user's workstation.
If the opened files are assigned a changelist, use 'p4 change -o nnn' to print out the changelist. It may indicate the purpose of the change.
Notify the manager and ask them to submit the files if needed. Do not delete the client until the manager is done.
If the manager does not want the change and the edits are assigned to a changelist, use 'p4 change -f -d nnn' to delete the changelist. You need 'open' or 'edit' permission on these files.
You should use 'p4 rename' if possible. An alternative is to directly edit the checkpoint file. The main problem is that it is easy to change too much or too little and end up with garbage.
See also
Paul Goffin PGoffin@baltimore.com
Mon, 16 Aug 1999 16:36:42 +0100
http://maillist.perforce.com/pipermail/perforce-user/1999-August/002864.html
> 1. Stop Perforce.
> 2. Take a checkpoint.
> 3. Run sed on the checkpoint file (it's a plain-text file, one
> database record per line).
> 4. Rebuild the database files from the checkpoint.
> 5. Move your depot directory so it has the correct name.
> 6. Restart Perforce.
>
> The hard part is step 3: performing the correct transformation on the
> checkpoint file. I don't know whether the web site or FTP site has a
> collection of suitable sed/awk/perl scripts (for applying common
> transformations on a repository) but it's something they could
> consider.
I'd use vi (or ed):
1,$s/\/\/DEPOT/\/\/AUDIO/g
:wq
Or, these days, I often use MS dev studio for this sort of
thing. Then I just do a find/replace of //DEPOT for //AUDIO
There is one more thing of concern. If the "DEPOT" is
actually the default depot "depot", then it may not
have been "created". So you'll need to add the following
line to the checkpoint:
@pv@ 0 @db.depot@ @AUDIO@ 0 @subdir@ @AUDIO/...@
(use your editor to search for "db.depot" if its not there, you'll
need the above. If it is there, it will have been
missed by my search & replace as the db.depot entry is the
only one that doesn't use the "//" prefix, so it'll
need to be changed anyway)
Paul
Paul D Goffin, Technical Leader, Process & Certification, Baltimore
Technologies Ltd.
Fredric Fredricson fredric.fredriksson@mydata.se
Wed, 23 Sep 1998 22:37:04 +0200
http://maillist.perforce.com/pipermail/perforce-user/1998-September/001400.html
There is a small problem with the solution suggested
in the technote:
The technote suggests a simple sed script to modify
the path in the depot. This means that if a change
description contains a file name in depot format
this will also be changed.
This *may* cause p4d to fail to read the modified
checkpoint file.
Paul Goffin PGoffin@baltimore.com
Thu, 10 Feb 2000 09:57:42 -0000
http://maillist.perforce.com/pipermail/perforce-user/2000-February/003701.html
If it's a real big rearangement, you
might want to consider the UNSUPPORTED
mechanism of directly editing the
metadata and moving the archive files around.
I do this sort of thing quite often - and
we don't have any problems from doing it.
But then, I'm VERY careful (I write
scripts to do the editing and then print
them out and go over them "with
a fine tooth comb" before actually
commiting the changes.)
Anyway, from the now defunct TechNote 20 (I expect because
users screwed this up too often):
"That way you move your history too.
To preserve all history but avoid an integration record
By editing a checkpoint and recovering from the checkpoint, you can preserve
your entire depot, but you won't have an integration record and you won't
need to save the old tree.
You'd probably use this method if you are about to go production with your
Perforce system, already have lots of changes you want to save, but realize
you'd like to change the structure.
Assume /src1/perforce is where you've installed p4d. (P4ROOT).
Take a checkpoint of your current depot.
p4d -r /src1/perforce -jc
Edit the checkpoint file and change all the places where the file says
//depot/file.c to //depot/main/file.c
You could write a sed script to do this if you wanted:
sed 's?//depot/core?//depot/main?g' new.checkpoint
Move the rcs files from under the /src1/perforce/depot/core to
/src1/perforce/depot/main.
Recover from your checkpoint:
rm /src1/perforce/db.*
p4d -jr new.checkpoint
Perforce maintains separate tables for users, have lists, changes, etc. There is a corresponding "db.*" file for each table. A checkpoint lists each row of the table with @pv@ [Put Value] and the table's revision number.
To produce a checkpoint for selected tables, follow Wally Dutchess's suggestion:
Copy the db.* files to a temp root directory, remove all db. files except the ones you want, then perform a checkpoint in the temp root -- bingo, a lowfat checkpoint.
MaxResults and MaxScanRows should be large enough to check out source files without sycning individual directories.
To increase the default value of MaxResults and MaxScanRows
cd geezer:/road/perforce/triggers
p4 edit p4d-watch-admin.pl
...modify $::maxResults or $::maxScanRows
p4 submit
cd /etc/init.d
sudo p4d-watch.d stop
sudo p4d-watch.d start
p4 edit //road/Perforce/main/protections/group/...
p4 submit
p4 group -o it
For introduction see Perforce's Tech Note 28. and System Admin: Triggers.
The Perforce databases need cleanup from time to time. Here are some items:
df
sleep 30;p4 label -d -f eisbock-3338
Delete all 300-day or older files: find . -mtime +300 -atime +300 -type f -exec sudo rm {} \;
p4 opened -a | sed 's/\(.*\) by \(.*\)/\2 \1/' | sort > opened.txt
Be careful. Use 'p4 -ztag' instead of 'p4 clients'. The later reports client updates while the former reports syncs. Look at clients before deleting them and check for opened files. The client may be active even though it has opened files on a retired codeline.
p4 -ztag clients | egrep '\.\.\. (client|Access|Owner)' >
clients.txt
TextPad Search: ... client \([-.0-9a-zA-Z_]*\)\n... Access
\([0-9]+\)\n... Owner \(-_[a-zA-Z]+\)
TextPad Replace: \3 \2 \1
sort +2 -n clients.txt
There should be a client report that gets e-mailed to each user. It can show clients sorted by date with instructions on how to delete clients on hosts that no longer exist.
From Michael Shields of Perforce
> > 21. Run the mapstate0.sh script. You'll find the mapstate0.sh script in the
> public depot at //guest/michael_shields/scripts/mapstate0.sh.
./mapstate0.sh | sed 's/.* [0-9]* //' | sort >mapstate.txt
Attempt to set mapstate for each of these clients
p4 -c road-nibbler-web fstat //user/xyz
...
Rerun mapstate0.sh
> > 22. Ask users with clients in the mapstate0.sh output to:
> > a. p4 sync #have (beware of opened files)
> > b. p4 fstat //depot/foo (can be //depot/foo, even if it doesn't exist)
The above steps identify which clients are misaligned with respect to the
client's view and it's have list. Users with clients in the resulting list
should (and would want to) align their client's view and it's have list. Step 22
a. will align the client's view and it's have list (provided that none of the
misaligned files are open), and step 22 b. will set the MapState flag now that
the client is aligned.
The mapstate0.sh script may need some customization for your environment: this
script is written to use an awk that understands strftime(), such as gawk. If
your awk does not understand strftime(), you'll need to remove the final pipe to
awk.
How many files? Only worry about
^\([^ ]+\) \([^ ]*\) ... MapState 0
echo \1 \2 `p4 -c \2 files //\2/...#have | wc -l`
Suggested e-mail:
Please realign your misaligned clients. See the list below. Jeanne, please
check the qa-script clients. Mary, please check the techsupport clients.
To realign client 'clientname'
Go to the client's host machine
p4 -c clientname sync #have
p4 -c clientname fstat //user/xyz
You can drop the "-c clientname" if it is your active client.
Why?
Whenever you edit a clientspec, Perforce's 'have' list of your files may not
match your clientspec. Perforce must scan your 'have' list when you use
depot syntax (e.g., //user/xyz). With 30,000 files, this is a slow operation that
blocks everyone's use of Perforce.
'p4 sync' or 'p4 sync #have' realigns your files with your clientspec.
'p4 fstat //user/xyz' (or any other use of depot syntax) scans your 'have' list
and verifies alignment. For details, see Michael Shield's FAQ on mapstate.
p4d -jd users.txt
egrep '@@ *$' users.txt
p4 -Ztag users | grep "... Email" | sed 's/... Email //' |
sort
All lists, show users
uniq -23 (to delete)
uniq -13 (to add, remove lionbridge, releng, perforce)
p4 -ztag clients | egrep '\.\.\. (client|Host|Owner)' >
clients.txt
TextPad Search:... client \([-.0-9a-zA-Z_]*\)\n... Owner
\([-a-zA-Z0-9]+\)\n... Host \([-.a-zA-X _]*$\)
TextPad Replace: \2 \1 @\3@
sort +2 -n clients.txt
Make a client of everything that you want obliterated. Double-check it! Then run 'p4 obliterate -y ...' The files and all record of the branch will go away.
I've requested an option to delete unrev'd files from branches.
Richard Baum of Perforce (reb@perforce.com) made the following recomendations in his 2001 User Conference paper [Solaris]:
From perforce/technotes/note040.html, compare the response time of (1) 'p4 info' and (2) p4win's "Show Connection Info". If both are slow, you have network problems. If (1) is slow but (2) is fast, you have DNS problems.
Review the event log, /home/perforce/logs/p4d. Notes
from Michael Shields of Perforce (CALL#306631):
Yes, it takes locks on a number of db.* tables, but releases the locks before transferring files from the client to the server. The general rule is that we avoid network I/O while we have locks on the db.* files.
It's interesting to note that it is during dm-SubmitChange that the transferring of files from the client to the server occurs. Perhaps you were transferring large files or there was a network anomaly?
There is no such thing as a "database lock". When a command starts it's compute phase, it locks only the tables that are needed, and only in the mode (read vs. write) that are needed. Most commands take read locks on either db.rev or db.have, so multiple readers can coexist with no problem. The challenge begins when a writer is introduced. We try to ensure that write locks are taken and released as quickly as possible. If a write lock is taken on either db.rev or db.have, the entire metadata is effectively locked.
Yes. See public.perforce.com:1666//guest/michael_shields/src/p4loga. It is a C++ program compiled with Jam.
See
p4 opened -a ... | sed -e 's/.* by //' -e 's/@.*//' | sort -u
Perforce scripting can be tricky since the client does not consistently handle errors.
The Bugzilla-Perforce interface uses a variety of techniques including the Perl API. See //road/Bugzilla/main/bugzilla/p4_diffs.cgi, perforce.pl, perldoc -t P4, perldoc P4::Client, and perldoc P4::UI.
export P4PORT=perforce.atg.com:1666
export P4CONFIG=p4config.txt
export P4USER=$USER
export P4CLIENT=$USER-$HOST-all
if [ -f .p4passwd ]; then
chmod 600 .p4passwd
export P4PASSWD=`cat .p4passwd`
fi
In the Perl API, see NewPerforceConnection() in //road/Bugzilla/main/bugzilla/perforce.pl. It sets the current working directory to the P4CONFIG file.
my $p4passwd = `sed -n '/^P4PASSWD=/s/P4PASSWD=//p'
$config_file` || die "Could not read P4PASSWD= from
$config_file";
$p4passwd =~ s/\s+//g;
`p4 -u releng -P $p4passwd ...`
The p4 client does not recognize 'cd' commands in a script. It uses the value of 'PWD' to set the current directory.
cd ${HOME}
PWD=`pwd`
export PWD
In order to lock all the labels in Perforce, I wrote a little 'one-liner' shell script in zsh:
for x in `p4 labels | awk '{print $2}'`
do
p4 label -o $x | sed 's/unlocked/locked/' | p4 label -i
done
This worked great, with the exception that every 10th or 15th label
would get a "Client Error: Can't connect".
A little investigation revealed that the Perforce server had about 9,000 TIME_WAIT sockets waiting to close, and so it couldn't allocate any more connections fast enough to satisfy the request which times out.
*SO* the solution was to add a minimal delay into the loop:
for x in `p4 labels | awk '{print $2}'`
do
p4 label -o $x | sed 's/unlocked/locked/' | (sleep 1;p4 label -i)
done
That little sleep 1 makes all the difference. Now every request goes through.
Use 'p4 -Ztag' to show individual fields of the response.
HKEY_CURRENT_USER\Software\Perforce\p4scc\TraceFlags
It must be a string value (REG_SZ) containing '*' to "log everything". The logging information is written to a file named "PerforcePlugin.log" in the "temp" directory. [Wally Dutchess, Perforce]
In most cases, each user and each host should have their own client spec. Perforce uses the client spec to distinguish who has what files.
The following note is from Jeff Bowles:
When I'm teaching the Perforce course, I say very specific things about sharing workspaces. What follows is mostly a technical set of guidelines, but at the end is a suggestion about when sharing a workspace might make sense and saying that "usually, it's not necessary."
The rule-of-thumb that I recommend is this:
If it's the same physical area on disk with the same pathname then you can consider it the same workspace. It doesn't matter if it's accessed by multiple machines or multiple users, so long as it's the same disk: serial number 129-123023--12A or whatever. Note that this disk should be mounted so that you can use the same pathname for the workspace root, e.g. M:\homer or /usr/marge/workspace.
Then I give examples:
If you have a removable ZIP drive in your E:\ drive at work, and you have a removable ZIP drive in your E:\ drive at home, and you wanna carry the cartridge back and forth, it can be the same workspace and therefore, the same workspace name: lisa.work or whatever. Why? Because it's the same physical disk, on the same platform type (oops, should've mentioned that), in the same pathname.If you have a removable ZIP drive in your E:\ drive at work, and you have a removable ZIP drive in your F:\ drive at home, and you wanna carry the cartridge back and forth, you cannot do it without making at least one change: the pathname to the workspace needs to be the same on both machines. So you might want to use the OS mechanism to refer to the ZIP drive as Z:\ on both machines: on Windows/NT, that would use the command ``subst Z: E:\ '' on one machine and ``subst Z: F:\ '' on the other.
If you have a fixed disk, e.g. your C: drive, and you wanna share it between two computers, you'll probably have to refer to your workspace using some shared-drive-letter like Z: or M: using that ``subst'' trick, to make it a shared workspace.
If the workspace will be shared between Unix and NT users, and you want to use 'samba' or NFS to do it, stop. My advice is don't do it. (Otherwise, if the Unix user does a 'p4 sync' they'll pull down files with Unix end-of-line, and the NT users using that shared disk will see the wrong end-of-line and ... well ... it can be done, but it's just borrowing trouble. Use separate workspaces.
I hasten to point out that recent Perforce releases include a "Host:" field in the client specification for a workspace that defaults to the machine you were on when you ran "p4 client". (I think this was done because it was too easy for me to masquerade as your client and run 'p4 sync' to pull things onto my own C: drive, but updating your workspaces "p4 have" data.) So if you're sharing a workspace between computers, you might need to remove this field.
The rule: if it's the same platform, pathname, and physical disk, then you can share the workspace.
Note that this rule is a technical one, not a recommendation of a procedure. For shared development, probably separate workspaces makes more sense. But for the once-every-blue-moon patches that release 1.2 requires (and you're developing release 7.0 now) it might make sense to share the 1.2 patch workspace. Your choice.
Jeff Bowles, Jan 2001.
In the Perforce root directory
find . -mindepth 2 -type d -iname '*.gz' for all
compressed binary revisionsfind . -mindepth 2 -type f -iname '*,v' for text
filesfind . -mindepth 2 -type f -iname '*,v' -not -name
"*'*" | xargs grep '\$ +\$' for all text files containing "$
$". This does not work for files with spaces or single quotes in
them.To shut down Perforce, use the following commands
su perforce
/work/perforce/p4d.crondaily
exit
p4 admin stop