Perforce Performance. Michael Shields Performance Lab Manager Perforce Software
|
|
|
- Alan Sullivan
- 6 years ago
- Views:
Transcription
1 Perforce Performance Michael Shields Performance Lab Manager Perforce Software
2 (in the spirit of
3 Research validity of outrageous claims for those s that "just don't seem right" Example: Claim: {etailer} suspending account until updated Status: False Urban legends, common fallacies, etc. Advertisements aplenty 2007 Perforce Software 3
4 Objectives Explore: metadata locking filesystem cache exclusionary entries embedded wildcards Expose misconceptions Reveal facts not (yet) common knowledge Discuss some Perforce Server internals 2007 Perforce Software 4
5 Perforce Metadata Claim: Perforce Server uses metadata stored in the db.* files Status:? 2007 Perforce Software 5
6 Perforce Metadata Claim: Perforce Server uses metadata stored in the db.* files Status: True 2007 Perforce Software 6
7 Perforce Metadata Claim: Perforce Server uses metadata stored in the db.* files Status: True For example: db.rev, db.change, db.integed, db.have, etc. Only used by server, not clients Database schema located at: also applicable to checkpoint and journal entries 2007 Perforce Software 7
8 Metadata and Concurrency Multiple commands processed simultaneously Each command must use: physically consistent metadata B-trees in each db.* file must be structurally correct e.g. writer not splitting pages a reader is traversing logically consistent metadata db.* files correctly contain related data e.g. db.rev* entries exist for submitted changelist Synchronization required for metadata access 2007 Perforce Software 8
9 Metadata Locking Claim: There is a single global lock controlling all metadata access Status:? 2007 Perforce Software 9
10 Metadata Locking Claim: There is a single global lock controlling all metadata access Status:? Claim: The single global lock is an exclusive lock, serializing all metadata access Status:? 2007 Perforce Software 10
11 Metadata Locking Claim: There is a single global lock controlling all metadata access Status: False Claim: The single global lock is an exclusive lock, serializing all metadata access Status: False 2007 Perforce Software 11
12 Metadata Locking Claim: There is a single global lock controlling all metadata access Status: False Claim: The single global lock is an exclusive lock, serializing all metadata access Status: False If true, these would severely limit concurrency 2007 Perforce Software 12
13 Metadata Locking Each command locks only the db.* files it needs A lock on a db.* file is either shared or exclusive shared lock allows access by others using a shared lock used when might read but won t write to the db.* file exclusive lock denies access by others used when might write to the db.* file 2007 Perforce Software 13
14 Observe Metadata Locking Use OS utilities to observe locking behavior e.g. strace on Linux, truss on FreeBSD and Solaris, and FileMon on Windows For example (on Linux): $ strace -etrace=open,flock,close p4d -f -p (system calls for server startup) Perforce Server starting... -f (single-threaded server) to simplify tracing trace only open(), flock(), and close() calls 2007 Perforce Software 14
15 Observe Metadata Locking From another session: $ p4 -p sync //depot/main/... //depot/branch1/... //depot/main/file1#1 - added as /p4client/depot/main/file1 //depot/main/file2#1 - added as /p4client/depot/main/file2 //depot/main/file3#1 - added as /p4client/depot/main/file3 //depot/branch1/file1#1 - added as /p4client/depot/branch1/file1 //depot/branch1/file2#1 - added as /p4client/depot/branch1/file2 //depot/branch1/file3#1 - added as /p4client/depot/branch1/file3 two arguments head revisions implied six files synced 2007 Perforce Software 15
16 Command Startup Phase open("journal", O_WRONLY O_APPEND O_CREAT O_LARGEFILE, 0666) = 5 open("db.depot", O_RDWR O_CREAT O_LARGEFILE, 0666) = 6 flock(6, LOCK_SH) = 0 flock(6, LOCK_UN) = 0 open("license", O_RDONLY O_LARGEFILE) = 7 close(7) = 0 open("db.group", O_RDWR O_CREAT O_LARGEFILE, 0666) = 7 flock(7, LOCK_SH) = 0 flock(7, LOCK_UN) = 0 open("db.protect", O_RDWR O_CREAT O_LARGEFILE, 0666) = 8 flock(8, LOCK_SH) = 0 flock(8, LOCK_UN) = 0 open("db.user", O_RDWR O_CREAT O_LARGEFILE, 0666) = 9 flock(9, LOCK_EX) (LOCK_SH in !) = 0 flock(9, LOCK_UN) = 0 open("db.domain", O_RDWR O_CREAT O_LARGEFILE, 0666) = 10 flock(10, LOCK_EX) (LOCK_SH in !) = 0 flock(10, LOCK_UN) = 0 open("db.view", O_RDWR O_CREAT O_LARGEFILE, 0666) = 11 flock(11, LOCK_SH) = 0 flock(11, LOCK_UN) = Perforce Software 16
17 Compute Phase for First Argument open("db.have", O_RDWR O_CREAT O_LARGEFILE, 0666) = 12 flock(12, LOCK_SH) = 0 open("db.resolve", O_RDWR O_CREAT O_LARGEFILE, 0666) = 13 flock(13, LOCK_SH) = 0 open("db.revdx", O_RDWR O_CREAT O_LARGEFILE, 0666) = 14 flock(14, LOCK_SH) = 0 open("db.revhx", O_RDWR O_CREAT O_LARGEFILE, 0666) = 15 flock(15, LOCK_SH) = 0 open("db.working", O_RDWR O_CREAT O_LARGEFILE, 0666) = 16 flock(16, LOCK_SH) = 0 flock(16, LOCK_UN) = 0 flock(15, LOCK_UN) = 0 flock(14, LOCK_UN) = 0 flock(12, LOCK_UN) = 0 flock(13, LOCK_UN) = 0 computes dataset passed on to next phase db.revhx and db.revdx locked instead of db.rev head revisions implied by argument 2007 Perforce Software 17
18 Execution Phase for First Argument open("depot/main/file1,v", O_RDONLY O_LARGEFILE) = 17 close(17) = 0 open("depot/main/file2,v", O_RDONLY O_LARGEFILE) = 17 close(17) = 0 open("depot/main/file3,v", O_RDONLY O_LARGEFILE) = 17 close(17) = 0 file content retrieved and sent to client no db.* file locks held for this example, db.have is not updated during this execution phase 2007 Perforce Software 18
19 Compute Phase for Second Argument flock(12, LOCK_SH) = 0 flock(13, LOCK_SH) = 0 flock(14, LOCK_SH) = 0 flock(15, LOCK_SH) = 0 flock(16, LOCK_SH) = 0 flock(16, LOCK_UN) = 0 flock(15, LOCK_UN) = 0 flock(14, LOCK_UN) = 0 flock(12, LOCK_UN) = 0 flock(13, LOCK_UN) = 0 for this example, compute phases are similar no additional db.* files used 2007 Perforce Software 19
20 Execution Phase for Second Argument open("depot/main/file1,v", O_RDONLY O_LARGEFILE) = 17 close(17) = 0 open("depot/main/file2,v", O_RDONLY O_LARGEFILE) = 17 close(17) = 0 open("depot/main/file3,v", O_RDONLY O_LARGEFILE) = 17 close(17) = 0 flock(12, LOCK_EX) = 0 flock(5, LOCK_EX) = 0 flock(5, LOCK_UN) = 0 flock(12, LOCK_UN) = 0 second argument was //depot/branch1/... lazy copies of //depot/main/... db.have (fd 12) updated for all six files synced during a single exclusive lock (journal is fd 5) batch updates introduced in extended to integrate in Perforce Software 20
21 Command Cleanup Phase close(4) = 0 close(5) = 0 close(9) = 0 close(7) = 0 close(6) = 0 close(10) = 0 close(11) = 0 close(12) = 0 close(13) = 0 close(14) = 0 close(15) = 0 close(16) = 0 close(8) = 0 other system calls not traced, e.g. fsync() 2007 Perforce Software 21
22 Other Observations Locks taken and released as needed not held over the entire command Type of db.* file lock varies throughout command shared or exclusive e.g. db.have Locks always taken in the same order eliminates possibility of deadlock simpler and faster server code 2007 Perforce Software 22
23 Review? Claim: A command holding only shared locks will not block another from acquiring a shared lock Status:? 2007 Perforce Software 23
24 Review? Claim: A command holding only shared locks will not block another from acquiring a shared lock Status: True 2007 Perforce Software 24
25 Review? Claim: A command holding only shared locks will not block another from acquiring a shared lock Status: True (directly) and false (indirectly) 2007 Perforce Software 25
26 "Wedge" Scenario Claim: A command holding only shared locks will not block another from acquiring a shared lock Status: True (directly) and false (indirectly) A writer can "wedge" between readers writer acquires at least one exclusive lock writer stalls acquiring an exclusive lock on a db.* file for which a shared lock is held by a reader other readers stall acquiring shared locks on db.* files for which exclusive locks are held by the writer Not a deadlock 2007 Perforce Software 26
27 Physical Memory Claim: If active server processes are swapped out, then additional physical memory is needed Status:? 2007 Perforce Software 27
28 Physical Memory Claim: If active server processes are swapped out, then additional physical memory is needed Status: True 2007 Perforce Software 28
29 Physical Memory Claim: If active server processes are swapped out, then additional physical memory is needed Status: True Claim: If active server processes are not swapped out, then physical memory is adequate Status:? 2007 Perforce Software 29
30 Physical Memory Claim: If active server processes are swapped out, then additional physical memory is needed Status: True Claim: If active server processes are not swapped out, then physical memory is adequate Status: Not necessarily true 2007 Perforce Software 30
31 Physical Memory Claim: If active server processes are swapped out, then additional physical memory is needed Status: True Claim: If active server processes are not swapped out, then physical memory is adequate Status: Not necessarily true For best performance, additional physical memory required for filesystem cache 2007 Perforce Software 31
32 Filesystem Cache Satisfy I/O requests without physical I/O logical I/O operations complete much faster, which generally speeds various server phases, which decreases time db.* file locks are held, which leads to better concurrency, which allows more commands processed faster 2007 Perforce Software 32
33 Filesystem Cache Content Consists of recently used pages from disk files significant portion will be pages from db.* files Metadata accessed by traversing B-tree only portions of the B-tree are read whole db.* files are generally not cached Portions of db.* files not cached unused client workspaces (portions of db.have) retired release branches (portions of db.rev*) etc Perforce Software 33
34 Perforce and Filesystem Cache Server relies heavily on filesystem cache no global cache within server less complex and more stable similar functionality offered by filesystem cache Server does cache within each command each db.* file opened has a private cache see "db.* File Generation Numbers" in 2006 European User Conference additional caching within server data structures 2007 Perforce Software 34
35 Example of Filesystem Cache Benefit Benchmark using Solaris 10 server on Opteron "forcedirectio" mount option simulates no cache Four concurrent "browse" clients on Linux 2.6 each "browse" issues dirs and fstat commands from the top to the bottom of the repository typical of drilling down using P4V each client does 64 "browses" without delay random; same seeds used for each run dedicated subnet 2007 Perforce Software 35
36 Total Elapsed Time for 256 Browses 2007 Perforce Software 36
37 Total Elapsed Time for 256 Browses 2007 Perforce Software 37
38 Total Elapsed Time for 256 Browses 2007 Perforce Software 38
39 Total Elapsed Time for 256 Browses 2007 Perforce Software 39
40 Filesystem Cache Effectiveness Goal is to minimize physical I/O for db.* files high level of physical I/O might signal the need for more physical memory Monitor physical I/O with OS utilities iostat on Linux, FreeBSD, and Solaris perfmon on Windows 2007 Perforce Software 40
41 Exclusionary Entries Claim: Exclusionary entries in protections and client views always improve performance Status:? 2007 Perforce Software 41
42 Exclusionary Entries Claim: Exclusionary entries in protections and client views always improve performance Status: False 2007 Perforce Software 42
43 Exclusionary Entries Claim: Exclusionary entries in protections and client views always improve performance Status: False Exclusionary entries: "-" starts depot path for example: list group dev * -//depot/product/main/doc/... Limiting files is recommended in the Perforce System Administrator's Guide type of entry has performance implications 2007 Perforce Software 43
44 Exclusionary Entries Can significantly impact dirs command more files scanned in the metadata General dirs behavior: position using non-exclusionary entries as directory is collected, skip to next directory Exclusionary entries necessitate scanning just to collect a directory must scan to find a directory that's not excluded Wider exclusionary scope yields more scanning subdirectories (recursively) included in scope 2007 Perforce Software 44
45 dirs Example p4 dirs //depot/product/main/* write group eng * //depot/product/main/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 45
46 dirs Example p4 dirs //depot/product/main/* write group eng * //depot/product/main/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 46
47 dirs Example p4 dirs //depot/product/main/* write group eng * //depot/product/main/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 47
48 dirs Example p4 dirs //depot/product/main/* write group eng * //depot/product/main/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc //depot/product/main/art 2007 Perforce Software 48
49 dirs Example p4 dirs //depot/product/main/* write group eng * //depot/product/main/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 49
50 dirs Example p4 dirs //depot/product/main/* write group eng * //depot/product/main/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 50
51 dirs Example p4 dirs //depot/product/main/* write group eng * //depot/product/main/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc //depot/product/main/doc 2007 Perforce Software 51
52 dirs Example p4 dirs //depot/product/main/* write group eng * //depot/product/main/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 52
53 dirs Example p4 dirs //depot/product/main/* write group eng * //depot/product/main/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 53
54 dirs Example p4 dirs //depot/product/main/* write group eng * //depot/product/main/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc //depot/product/main/src 2007 Perforce Software 54
55 dirs Example p4 dirs //depot/product/main/* write group eng * //depot/product/main/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 55
56 dirs Example p4 dirs //depot/product/main/* write group eng * //depot/product/main/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 56
57 With Protection Exclusions p4 dirs //depot/product/main/* write group eng * //depot/product/main/... list group dev * -//depot/product/main/doc/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 57
58 With Protection Exclusions p4 dirs //depot/product/main/* write group eng * //depot/product/main/... list group dev * -//depot/product/main/doc/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 58
59 With Protection Exclusions p4 dirs //depot/product/main/* write group eng * //depot/product/main/... list group dev * -//depot/product/main/doc/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 59
60 With Protection Exclusions p4 dirs //depot/product/main/* write group eng * //depot/product/main/... list group dev * -//depot/product/main/doc/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc //depot/product/main/art 2007 Perforce Software 60
61 With Protection Exclusions p4 dirs //depot/product/main/* write group eng * //depot/product/main/... list group dev * -//depot/product/main/doc/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 61
62 With Protection Exclusions p4 dirs //depot/product/main/* write group eng * //depot/product/main/... list group dev * -//depot/product/main/doc/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 62
63 With Protection Exclusions p4 dirs //depot/product/main/* write group eng * //depot/product/main/... list group dev * -//depot/product/main/doc/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 63
64 With Protection Exclusions p4 dirs //depot/product/main/* write group eng * //depot/product/main/... list group dev * -//depot/product/main/doc/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 64
65 With Protection Exclusions p4 dirs //depot/product/main/* write group eng * //depot/product/main/... list group dev * -//depot/product/main/doc/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 65
66 With Protection Exclusions p4 dirs //depot/product/main/* write group eng * //depot/product/main/... list group dev * -//depot/product/main/doc/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc //depot/product/main/src 2007 Perforce Software 66
67 With Protection Exclusions p4 dirs //depot/product/main/* write group eng * //depot/product/main/... list group dev * -//depot/product/main/doc/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 67
68 With Protection Exclusions p4 dirs //depot/product/main/* write group eng * //depot/product/main/... list group dev * -//depot/product/main/doc/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 68
69 With Protection and View Exclusions p4 dirs -C //depot/product/main/* //depot/product/main/... //dev-client/depot/product/main/... -//depot/product/main/art/... //dev-client/depot/product/main/art/... write group eng * //depot/product/main/... list group dev * -//depot/product/main/doc/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 69
70 With Protection and View Exclusions p4 dirs -C //depot/product/main/* //depot/product/main/... //dev-client/depot/product/main/... -//depot/product/main/art/... //dev-client/depot/product/main/art/... write group eng * //depot/product/main/... list group dev * -//depot/product/main/doc/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 70
71 With Protection and View Exclusions p4 dirs -C //depot/product/main/* //depot/product/main/... //dev-client/depot/product/main/... -//depot/product/main/art/... //dev-client/depot/product/main/art/... write group eng * //depot/product/main/... list group dev * -//depot/product/main/doc/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 71
72 With Protection and View Exclusions p4 dirs -C //depot/product/main/* //depot/product/main/... //dev-client/depot/product/main/... -//depot/product/main/art/... //dev-client/depot/product/main/art/... write group eng * //depot/product/main/... list group dev * -//depot/product/main/doc/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 72
73 With Protection and View Exclusions p4 dirs -C //depot/product/main/* //depot/product/main/... //dev-client/depot/product/main/... -//depot/product/main/art/... //dev-client/depot/product/main/art/... write group eng * //depot/product/main/... list group dev * -//depot/product/main/doc/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 73
74 With Protection and View Exclusions p4 dirs -C //depot/product/main/* //depot/product/main/... //dev-client/depot/product/main/... -//depot/product/main/art/... //dev-client/depot/product/main/art/... write group eng * //depot/product/main/... list group dev * -//depot/product/main/doc/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 74
75 With Protection and View Exclusions p4 dirs -C //depot/product/main/* //depot/product/main/... //dev-client/depot/product/main/... -//depot/product/main/art/... //dev-client/depot/product/main/art/... write group eng * //depot/product/main/... list group dev * -//depot/product/main/doc/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 75
76 With Protection and View Exclusions p4 dirs -C //depot/product/main/* //depot/product/main/... //dev-client/depot/product/main/... -//depot/product/main/art/... //dev-client/depot/product/main/art/... write group eng * //depot/product/main/... list group dev * -//depot/product/main/doc/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 76
77 With Protection and View Exclusions p4 dirs -C //depot/product/main/* //depot/product/main/... //dev-client/depot/product/main/... -//depot/product/main/art/... //dev-client/depot/product/main/art/... write group eng * //depot/product/main/... list group dev * -//depot/product/main/doc/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 77
78 With Protection and View Exclusions p4 dirs -C //depot/product/main/* //depot/product/main/... //dev-client/depot/product/main/... -//depot/product/main/art/... //dev-client/depot/product/main/art/... write group eng * //depot/product/main/... list group dev * -//depot/product/main/doc/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc //depot/product/main/src 2007 Perforce Software 78
79 With Protection and View Exclusions p4 dirs -C //depot/product/main/* //depot/product/main/... //dev-client/depot/product/main/... -//depot/product/main/art/... //dev-client/depot/product/main/art/... write group eng * //depot/product/main/... list group dev * -//depot/product/main/doc/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 79
80 With Protection and View Exclusions p4 dirs -C //depot/product/main/* //depot/product/main/... //dev-client/depot/product/main/... -//depot/product/main/art/... //dev-client/depot/product/main/art/... write group eng * //depot/product/main/... list group dev * -//depot/product/main/doc/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 80
81 Enumerated Inclusionary Protections p4 dirs //depot/product/main/* write group dev * //depot/product/main/art/... write group dev * //depot/product/main/src/... write group doc * //depot/product/main/... write group qa * //depot/product/main/... write group rel * //depot/product/main/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 81
82 Enumerated Inclusionary Protections p4 dirs //depot/product/main/* write group dev * //depot/product/main/art/... write group dev * //depot/product/main/src/... write group doc * //depot/product/main/... write group qa * //depot/product/main/... write group rel * //depot/product/main/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 82
83 Enumerated Inclusionary Protections p4 dirs //depot/product/main/* write group dev * //depot/product/main/art/... write group dev * //depot/product/main/src/... write group doc * //depot/product/main/... write group qa * //depot/product/main/... write group rel * //depot/product/main/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 83
84 Enumerated Inclusionary Protections p4 dirs //depot/product/main/* write group dev * //depot/product/main/art/... write group dev * //depot/product/main/src/... write group doc * //depot/product/main/... write group qa * //depot/product/main/... write group rel * //depot/product/main/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc //depot/product/main/art 2007 Perforce Software 84
85 Enumerated Inclusionary Protections p4 dirs //depot/product/main/* write group dev * //depot/product/main/art/... write group dev * //depot/product/main/src/... write group doc * //depot/product/main/... write group qa * //depot/product/main/... write group rel * //depot/product/main/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 85
86 Enumerated Inclusionary Protections p4 dirs //depot/product/main/* write group dev * //depot/product/main/art/... write group dev * //depot/product/main/src/... write group doc * //depot/product/main/... write group qa * //depot/product/main/... write group rel * //depot/product/main/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 86
87 Enumerated Inclusionary Protections p4 dirs //depot/product/main/* write group dev * //depot/product/main/art/... write group dev * //depot/product/main/src/... write group doc * //depot/product/main/... write group qa * //depot/product/main/... write group rel * //depot/product/main/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc //depot/product/main/src 2007 Perforce Software 87
88 Enumerated Inclusionary Protections p4 dirs //depot/product/main/* write group dev * //depot/product/main/art/... write group dev * //depot/product/main/src/... write group doc * //depot/product/main/... write group qa * //depot/product/main/... write group rel * //depot/product/main/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 88
89 Enumerated Inclusionary Protections p4 dirs //depot/product/main/* write group dev * //depot/product/main/art/... write group dev * //depot/product/main/src/... write group doc * //depot/product/main/... write group qa * //depot/product/main/... write group rel * //depot/product/main/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 89
90 Enumerated Inclusionary Protections and Views p4 dirs -C //depot/product/main/* //depot/product/main/doc/... //dev-client/depot/product/main/doc/... //depot/product/main/src/... //dev-client/depot/product/main/src/... write group dev * //depot/product/main/art/... write group dev * //depot/product/main/src/... write group doc * //depot/product/main/... write group qa * //depot/product/main/... write group rel * //depot/product/main/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 90
91 Enumerated Inclusionary Protections and Views p4 dirs -C //depot/product/main/* //depot/product/main/doc/... //dev-client/depot/product/main/doc/... //depot/product/main/src/... //dev-client/depot/product/main/src/... write group dev * //depot/product/main/art/... write group dev * //depot/product/main/src/... write group doc * //depot/product/main/... write group qa * //depot/product/main/... write group rel * //depot/product/main/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 91
92 Enumerated Inclusionary Protections and Views p4 dirs -C //depot/product/main/* //depot/product/main/doc/... //dev-client/depot/product/main/doc/... //depot/product/main/src/... //dev-client/depot/product/main/src/... write group dev * //depot/product/main/art/... write group dev * //depot/product/main/src/... write group doc * //depot/product/main/... write group qa * //depot/product/main/... write group rel * //depot/product/main/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 92
93 Enumerated Inclusionary Protections and Views p4 dirs -C //depot/product/main/* //depot/product/main/doc/... //dev-client/depot/product/main/doc/... //depot/product/main/src/... //dev-client/depot/product/main/src/... write group dev * //depot/product/main/art/... write group dev * //depot/product/main/src/... write group doc * //depot/product/main/... write group qa * //depot/product/main/... write group rel * //depot/product/main/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc //depot/product/main/src 2007 Perforce Software 93
94 Enumerated Inclusionary Protections and Views p4 dirs -C //depot/product/main/* //depot/product/main/doc/... //dev-client/depot/product/main/doc/... //depot/product/main/src/... //dev-client/depot/product/main/src/... write group dev * //depot/product/main/art/... write group dev * //depot/product/main/src/... write group doc * //depot/product/main/... write group qa * //depot/product/main/... write group rel * //depot/product/main/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 94
95 Enumerated Inclusionary Protections and Views p4 dirs -C //depot/product/main/* //depot/product/main/doc/... //dev-client/depot/product/main/doc/... //depot/product/main/src/... //dev-client/depot/product/main/src/... write group dev * //depot/product/main/art/... write group dev * //depot/product/main/src/... write group doc * //depot/product/main/... write group qa * //depot/product/main/... write group rel * //depot/product/main/... //depot/product/main/art/file1.jpg file2.jpg file3.jpg //depot/product/main/doc/file1.fm file2.fm file3.fm //depot/product/main/src/file1.cc file2.cc file3.cc 2007 Perforce Software 95
96 Embedded Wildcards Claim: For best performance, protections and client views must be concise Status:? 2007 Perforce Software 96
97 Embedded Wildcards Claim: For best performance, protections and client views must be concise Status: False 2007 Perforce Software 97
98 Embedded Wildcards Claim: For best performance, protections and client views must be concise Status: False Example usage: write group doc * //depot/.../doc/... Results are typically more concise might be easier to understand Can require additional CPU and I/O when protections table joined with client view 2007 Perforce Software 98
99 Minimize Embedded Wildcards Repository structure not consistent with usage examples: documentation scattered within source platform-specific scattered within generic Restructuring might be better solution, or Perforce Software 99
100 Enumerating Entries Replace with explicitly enumerated entries provides same (dataset-specific) functionality allows for "fine-tuning" Typically increases size increased protection table size tolerated well increased client view size perhaps not so well Performance improvement can be significant following example for sync compute phase 7000 files returned 2007 Perforce Software 100
101 CPU Time 2007 Perforce Software 101
102 CPU Time 2007 Perforce Software 102
103 CPU Time 2007 Perforce Software 103
104 CPU Time 2007 Perforce Software 104
105 CPU Time 2007 Perforce Software 105
106 db.revhx Reads 2007 Perforce Software 106
107 db.revhx Reads 2007 Perforce Software 107
108 db.revhx Reads 2007 Perforce Software 108
109 db.revhx Reads 2007 Perforce Software 109
110 db.revhx Reads 2007 Perforce Software 110
111 Example Protection Table Entries Using embedded wildcards: write group dev * //depot/main/.../00/... write group dev * //depot/main/.../01/ write group dev * //depot/main/.../08/... write group dev * //depot/main/.../09/... Using enumerated entries: write group dev * //depot/main/00/00/... write group dev * //depot/main/00/01/ write group dev * //depot/main/00/08/... write group dev * //depot/main/00/09/... write group dev * //depot/main/01/00/... write group dev * //depot/main/01/01/ write group dev * //depot/main/99/08/... write group dev * //depot/main/99/09/ Perforce Software 111
112 Example Client View Entries Using embedded wildcards: //depot/main/.../00/... //dev-client/depot/main/.../00/... Using enumerated entries: //depot/main/00/00/... //dev-client/depot/main/00/00/... //depot/main/01/00/... //dev-client/depot/main/01/00/ //depot/main/98/00/... //dev-client/depot/main/98/00/... //depot/main/99/00/... //dev-client/depot/main/99/00/ Perforce Software 112
113 Summary Claim: Perforce Server uses metadata stored in the db.* files Status: True Claim: There is a single global lock controlling all metadata access Status: False 2007 Perforce Software 113
114 Summary Claim: The single global lock is an exclusive lock, serializing all metadata access Status: False Claim: A command holding only shared locks will not block another from acquiring a shared lock Status: True (directly) and false (indirectly) 2007 Perforce Software 114
115 Summary Claim: If active server processes are swapped out, then additional physical memory is needed Status: True Claim: If active server processes are not swapped out, then physical memory is adequate Status: Not necessarily true 2007 Perforce Software 115
116 Summary Claim: Exclusionary entries in protections and client views always improves performance Status: False Claim: For best performance, protections and client views must be concise Status: False 2007 Perforce Software 116
117 Questions?
Perforce Tunables. Michael Shields Performance Lab Manager Perforce Software. March, Abstract
Perforce Tunables Michael Shields Performance Lab Manager Perforce Software March, 2010 Abstract Tunables have been available in Perforce since the 2008.2 release. With help from Perforce, tunables can
Repository Structure Considerations for Performance
Repository Structure Considerations for Performance Perforce Software Performance Lab Michael Shields, Manager Tim Brazil, Engineer Introduction Two of many repository structure decisions path length placement
Performance and Database Locking at Large Perforce Sites
Performance and Database Locking at Large Perforce Sites Dan Bloch Sep. 19, 2006 Google Google's mission: Organize the world's information and make it universally accessible and useful. The world's premier
Repository Structure Considerations for Performance
Repository Structure Considerations for Performance Michael Shields Performance Lab Manager Perforce Software Tim Brazil Performance Lab Engineer Perforce Software March, 2009 Abstract The structure of
Life on the Edge: Monitoring and Running A Very Large Perforce Installation
Life on the Edge: Monitoring and Running A Very Large Perforce Installation Dan Bloch May 10, 2007 Google Google's mission: Organize the world's information and make it universally accessible and useful.
Better Living Through New Releases
Better Living Through New Releases by Richard E. Baum Overview Server performance-related functionality Upgrades contain much more than bug fixes This is a high level view Close to chronological order
Still All on One Server: Perforce at Scale
Still All on One Server: Perforce at Scale Dan Bloch Senior Site Reliability Engineer Google Inc. June 3, 2011 GOOGLE Google's mission: Organize the world's information and make it universally accessible
Staying Out of the Swamp
Staying Out of the Swamp Perforce User Conference 2001 Richard E. Baum Introduction Perforce runs well when given proper resources. CPU requirements are quite small. A server s I/O bandwidth is generally
PROJECT 6: PINTOS FILE SYSTEM. CS124 Operating Systems Winter , Lecture 25
PROJECT 6: PINTOS FILE SYSTEM CS124 Operating Systems Winter 2015-2016, Lecture 25 2 Project 6: Pintos File System Last project is to improve the Pintos file system Note: Please ask before using late tokens
P4EXP Help January 2018
P4EXP Help 2018.1 January 2018 Copyright 1999-2018 Perforce Software. All rights reserved. Perforce Software and documentation is available from www.perforce.com. You can download and use Perforce programs,
Filesystem Performance on FreeBSD
Filesystem Performance on FreeBSD Kris Kennaway kris@freebsd.org BSDCan 2006, Ottawa, May 12 Introduction Filesystem performance has many aspects No single metric for quantifying it I will focus on aspects
Improving Perforce Performance At Research In Motion (RIM)
Improving Perforce Performance At Research In Motion (RIM) Perforce User Conference il 2008 Tim Barrett 1 Agenda RIM s Environment RIM s Perforce performance problems Project approach and major initiatives
P4Admin User Guide October 2017
P4Admin User Guide 2017.3 October 2017 Copyright 1999-2017 Perforce Software. All rights reserved. Perforce Software and documentation is available from www.perforce.com. You can download and use Perforce
Still All on One Server: Perforce at Scale
Still All on One Server: Perforce at Scale Dan Bloch, Google Inc. May 21, 2011 Abstract Google runs the busiest single Perforce server on the planet, and one of the largest repositories in any source control
Scaling Source Control for the Next Generation of Game Development
Scaling Source Control for the Next Generation of Game Development Mike Sundy Toby Roberts 2007 Perforce User s Conference Scaling Source Control for NextGen Game Development 1 Overview Current Gen vs.
Concurrency, Mutual Exclusion and Synchronization C H A P T E R 5
Concurrency, Mutual Exclusion and Synchronization C H A P T E R 5 Multiple Processes OS design is concerned with the management of processes and threads: Multiprogramming Multiprocessing Distributed processing
P4EXP Help July 2018
P4EXP Help 2018.3 July 2018 Copyright 1999-2018 Perforce Software. All rights reserved. Perforce Software and documentation is available from www.perforce.com. You can download and use Perforce programs,
CA485 Ray Walshe Google File System
Google File System Overview Google File System is scalable, distributed file system on inexpensive commodity hardware that provides: Fault Tolerance File system runs on hundreds or thousands of storage
CS 471 Operating Systems. Yue Cheng. George Mason University Fall 2017
CS 471 Operating Systems Yue Cheng George Mason University Fall 2017 Review: RAID 2 RAID o Idea: Build an awesome disk from small, cheap disks o Metrics: Capacity, performance, reliability 3 RAID o Idea:
Proposed Wording for Concurrent Data Structures: Hazard Pointer and Read Copy Update (RCU)
Document number: D0566R1 Date: 20170619 (pre Toronto) Project: Programming Language C++, WG21, SG1,SG14, LEWG, LWG Authors: Michael Wong, Maged M. Michael, Paul McKenney, Geoffrey Romer, Andrew Hunter
Operating Systems CMPSCI 377 Spring Mark Corner University of Massachusetts Amherst
Operating Systems CMPSCI 377 Spring 2017 Mark Corner University of Massachusetts Amherst Clicker Question #1 For a sequential workload, the limiting factor for a disk system is likely: (A) The speed of
Chapter 18: Database System Architectures.! Centralized Systems! Client--Server Systems! Parallel Systems! Distributed Systems!
Chapter 18: Database System Architectures! Centralized Systems! Client--Server Systems! Parallel Systems! Distributed Systems! Network Types 18.1 Centralized Systems! Run on a single computer system and
CS2028 -UNIX INTERNALS
DHANALAKSHMI SRINIVASAN INSTITUTE OF RESEARCH AND TECHNOLOGY,SIRUVACHUR-621113. CS2028 -UNIX INTERNALS PART B UNIT 1 1. Explain briefly details about History of UNIX operating system? In 1965, Bell Telephone
Lecture 11: Linux ext3 crash recovery
6.828 2011 Lecture 11: Linux ext3 crash recovery topic crash recovery crash may interrupt a multi-disk-write operation leave file system in an unuseable state most common solution: logging last lecture:
Chapter 11: File-System Interface
Chapter 11: File-System Interface Silberschatz, Galvin and Gagne 2013 Chapter 11: File-System Interface File Concept Access Methods Disk and Directory Structure File-System Mounting File Sharing Protection
Tricky issues in file systems
Tricky issues in file systems Taylor Riastradh Campbell campbell@mumble.net riastradh@netbsd.org EuroBSDcon 2015 Stockholm, Sweden October 4, 2015 What is a file system? Standard Unix concept: hierarchy
Advanced file systems: LFS and Soft Updates. Ken Birman (based on slides by Ben Atkin)
: LFS and Soft Updates Ken Birman (based on slides by Ben Atkin) Overview of talk Unix Fast File System Log-Structured System Soft Updates Conclusions 2 The Unix Fast File System Berkeley Unix (4.2BSD)
Perforce Getting Started with P4V
Perforce 2005.2 Getting Started with P4V December 2005 This manual copyright 2005 Perforce Software. All rights reserved. Perforce software and documentation is available from http://www.perforce.com.
CSCI 4717 Computer Architecture
CSCI 4717/5717 Computer Architecture Topic: Symmetric Multiprocessors & Clusters Reading: Stallings, Sections 18.1 through 18.4 Classifications of Parallel Processing M. Flynn classified types of parallel
FILE SYSTEMS, PART 2. CS124 Operating Systems Winter , Lecture 24
FILE SYSTEMS, PART 2 CS124 Operating Systems Winter 2015-2016, Lecture 24 2 Files and Processes The OS maintains a buffer of storage blocks in memory Storage devices are often much slower than the CPU;
CPSC/ECE 3220 Fall 2017 Exam Give the definition (note: not the roles) for an operating system as stated in the textbook. (2 pts.
CPSC/ECE 3220 Fall 2017 Exam 1 Name: 1. Give the definition (note: not the roles) for an operating system as stated in the textbook. (2 pts.) Referee / Illusionist / Glue. Circle only one of R, I, or G.
Data Validation Option Best Practices
Data Validation Option Best Practices 1993-2016 Informatica LLC. No part of this document may be reproduced or transmitted in any form, by any means (electronic, photocopying, recording or otherwise) without
Perforce System Administrator s Guide
Perforce 2007.2 System Administrator s Guide May 2007 This manual copyright 1997-2007 Perforce Software. All rights reserved. Perforce software and documentation is available from http://www.perforce.com.
CONTENTS. Computer-System Structures
CONTENTS PART ONE OVERVIEW Chapter 1 Introduction 1.1 What Is an Operating System? 3 1.2 Simple Batch Systems 6 1.3 Multiprogrammed Batched Systems 8 1.4 Time-Sharing Systems 9 1.5 Personal-Computer Systems
File Control System 1.0 Product Requirements Document (PRD)
File Control System 1.0 Product Requirements Document (PRD) Author: Ken Rodham Date: January 10, 2005 Revision: 2 Overview This document specifies the requirements for the File Control System 1.0 (FCS).
To Everyone... iii To Educators... v To Students... vi Acknowledgments... vii Final Words... ix References... x. 1 ADialogueontheBook 1
Contents To Everyone.............................. iii To Educators.............................. v To Students............................... vi Acknowledgments........................... vii Final Words..............................
Java Concurrency in practice Chapter 9 GUI Applications
Java Concurrency in practice Chapter 9 GUI Applications INF329 Spring 2007 Presented by Stian and Eirik 1 Chapter 9 GUI Applications GUI applications have their own peculiar threading issues To maintain
FILE SYSTEMS, PART 2. CS124 Operating Systems Fall , Lecture 24
FILE SYSTEMS, PART 2 CS124 Operating Systems Fall 2017-2018, Lecture 24 2 Last Time: File Systems Introduced the concept of file systems Explored several ways of managing the contents of files Contiguous
MS-FSSHTTP. Simon Clarke, SharePoint/OneDrive
MS-FSSHTTP Simon Clarke, SharePoint/OneDrive Agenda What is MS-FSSHTTP? Requests File Model and Co-auth Storage/Sync Model Storage Graph Object Graph Application Schema Sync Example The documents (MS-FSSHTTP,
Analyzing File Content History
Analyzing File Content History James Creasy Perforce Software Abstract: Have you ever needed to figure out what broke the build? Or needed to track down when, where, and why a particular line of code was
Optimizing MySQL performance with ZFS. Neelakanth Nadgir Allan Packer Sun Microsystems
Optimizing MySQL performance with ZFS Neelakanth Nadgir Allan Packer Sun Microsystems Who are we? Allan Packer Principal Engineer, Performance http://blogs.sun.com/allanp Neelakanth Nadgir Senior Engineer,
Chapter 6. Buffer cache. Code: Data structures
DRAFT as of October 16, 2009: Copyright 2009 Cox, Kaashoek, Morris Chapter 6 Buffer cache One of an operating system s central roles is to enable safe cooperation between processes sharing a computer.
Chapter 20: Database System Architectures
Chapter 20: Database System Architectures Chapter 20: Database System Architectures Centralized and Client-Server Systems Server System Architectures Parallel Systems Distributed Systems Network Types
STEPS Towards Cache-Resident Transaction Processing
STEPS Towards Cache-Resident Transaction Processing Stavros Harizopoulos joint work with Anastassia Ailamaki VLDB 2004 Carnegie ellon CPI OLTP workloads on modern CPUs 6 4 2 L2-I stalls L2-D stalls L1-I
P4VS User Guide Patch October 2017
P4VS User Guide 2017.2 Patch October 2017 Copyright 2012-2017 Perforce Software. All rights reserved. Perforce Software and documentation is available from www.perforce.com. You can download and use Perforce
Chapter 7: Deadlocks. Operating System Concepts 9 th Edition
Chapter 7: Deadlocks Silberschatz, Galvin and Gagne 2013 Chapter 7: Deadlocks System Model Deadlock Characterization Methods for Handling Deadlocks Deadlock Prevention Deadlock Avoidance Deadlock Detection
Module 6: Process Synchronization. Operating System Concepts with Java 8 th Edition
Module 6: Process Synchronization 6.1 Silberschatz, Galvin and Gagne 2009 Module 6: Process Synchronization Background The Critical-Section Problem Peterson s Solution Synchronization Hardware Semaphores
Chapter 11: File-System Interface
Chapter 11: File-System Interface Chapter 11: File-System Interface File Concept Access Methods Disk and Directory Structure File-System Mounting File Sharing Protection Objectives To explain the function
Seminar 3. Transactions. Concurrency Management in MS SQL Server
Seminar 3 Transactions Concurrency Management in MS SQL Server Transactions in SQL Server SQL Server uses transactions to compose multiple operations in a single unit of work. Each user's work is processed
1 of 8 14/12/2013 11:51 Tuning long-running processes Contents 1. Reduce the database size 2. Balancing the hardware resources 3. Specifying initial DB2 database settings 4. Specifying initial Oracle database
CSE 120. Fall Lecture 8: Scheduling and Deadlock. Keith Marzullo
CSE 120 Principles of Operating Systems Fall 2007 Lecture 8: Scheduling and Deadlock Keith Marzullo Aministrivia Homework 2 due now Next lecture: midterm review Next Tuesday: midterm 2 Scheduling Overview
CMP-3440 Database Systems
CMP-3440 Database Systems Concurrency Control with Locking, Serializability, Deadlocks, Database Recovery Management Lecture 10 zain 1 Basic Recovery Facilities Backup Facilities: provides periodic backup
Introduction to Data Management CSE 344
Introduction to Data Management CSE 344 Lecture 21: Transaction Implementations CSE 344 - Winter 2017 1 Announcements WQ7 and HW7 are out Due next Mon and Wed Start early, there is little time! CSE 344
CSE 120 Principles of Operating Systems
CSE 120 Principles of Operating Systems Fall 2016 Lecture 8: Scheduling and Deadlock Geoffrey M. Voelker Administrivia Thursday Friday Monday Homework #2 due at start of class Review material for midterm
PDSE Performance Basics
PDSE Performance Basics Speaker: Thomas Reed /IBM Corporation SHARE San Francisco Session:12982 (C) 2012, 2013 IBM Corporation Agenda System and SYSPLEX Level Performance PDSE Sharing PDSE Features PDSE
Concurrency: Deadlock and Starvation
Concurrency: Deadlock and Starvation Chapter 6 E&CE 354: Processes 1 Deadlock Deadlock = situation in which every process from a set is permanently blocked, i.e. cannot proceed with execution Common cause:
CSL373/CSL633 Major Exam Operating Systems Sem II, May 6, 2013 Answer all 8 questions Max. Marks: 56
CSL373/CSL633 Major Exam Operating Systems Sem II, 2012 13 May 6, 2013 Answer all 8 questions Max. Marks: 56 1. True or False. Give reasons and/or brief explanation. No marks for incomplete/wrong explanation.
Sistemi in Tempo Reale
Laurea Specialistica in Ingegneria dell'automazione Sistemi in Tempo Reale Giuseppe Lipari Introduzione alla concorrenza Fundamentals Algorithm: It is the logical procedure to solve a certain problem It
Using Helix Server for Distributed Versioning
Using Helix Server for Distributed Versioning 2017.2 October 2017 Copyright 2015-2018 Perforce Software. All rights reserved. Perforce Software and documentation is available from www.perforce.com. You
QUESTION BANK UNIT I
QUESTION BANK Subject Name: Operating Systems UNIT I 1) Differentiate between tightly coupled systems and loosely coupled systems. 2) Define OS 3) What are the differences between Batch OS and Multiprogramming?
Perforce Best Practices for Codeline Management. Organizing Codelines, Part 1
Perforce Best Practices for Codeline Management Organizing Codelines, Part 1 Your objectives for this exercise: Review the terminology used in defining codelines. Evaluate how codelines differ in their
Outline. Failure Types
Outline Database Tuning Nikolaus Augsten University of Salzburg Department of Computer Science Database Group 1 Unit 10 WS 2013/2014 Adapted from Database Tuning by Dennis Shasha and Philippe Bonnet. Nikolaus
Chapter 11: File-System Interface. Operating System Concepts 9 th Edition
Chapter 11: File-System Interface Silberschatz, Galvin and Gagne 2013 Chapter 11: File-System Interface File Concept Access Methods Disk and Directory Structure File-System Mounting File Sharing Protection
Last Class: Monitors. Real-world Examples
Last Class: Monitors Monitor wraps operations with a mutex Condition variables release mutex temporarily C++ does not provide a monitor construct, but monitors can be implemented by following the monitor
Fall 2015 COMP Operating Systems. Lab 06
Fall 2015 COMP 3511 Operating Systems Lab 06 Outline Monitor Deadlocks Logical vs. Physical Address Space Segmentation Example of segmentation scheme Paging Example of paging scheme Paging-Segmentation
Optimizing Testing Performance With Data Validation Option
Optimizing Testing Performance With Data Validation Option 1993-2016 Informatica LLC. No part of this document may be reproduced or transmitted in any form, by any means (electronic, photocopying, recording
Deadlock. Concurrency: Deadlock and Starvation. Reusable Resources
Concurrency: Deadlock and Starvation Chapter 6 Deadlock Permanent blocking of a set of processes that either compete for system resources or communicate with each other No efficient solution Involve conflicting
Perforce System Administrator s Guide
Perforce 2009.2 System Administrator s Guide December 2009 This manual copyright 1997-2009 Perforce Software. All rights reserved. Perforce software and documentation is available from http://www.perforce.com.
Advanced Topic: Efficient Synchronization
Advanced Topic: Efficient Synchronization Multi-Object Programs What happens when we try to synchronize across multiple objects in a large program? Each object with its own lock, condition variables Is
TxFS: Leveraging File-System Crash Consistency to Provide ACID Transactions
TxFS: Leveraging File-System Crash Consistency to Provide ACID Transactions Yige Hu, Zhiting Zhu, Ian Neal, Youngjin Kwon, Tianyu Chen, Vijay Chidambaram, Emmett Witchel The University of Texas at Austin
Outline. Database Tuning. Ideal Transaction. Concurrency Tuning Goals. Concurrency Tuning. Nikolaus Augsten. Lock Tuning. Unit 8 WS 2013/2014
Outline Database Tuning Nikolaus Augsten University of Salzburg Department of Computer Science Database Group 1 Unit 8 WS 2013/2014 Adapted from Database Tuning by Dennis Shasha and Philippe Bonnet. Nikolaus
CSE 333 SECTION 3. POSIX I/O Functions
CSE 333 SECTION 3 POSIX I/O Functions Administrivia Questions (?) HW1 Due Tonight Exercise 7 due Monday (out later today) POSIX Portable Operating System Interface Family of standards specified by the
DiskBoss DATA MANAGEMENT
DiskBoss DATA MANAGEMENT File Synchronization Version 9.1 Apr 2018 www.diskboss.com info@flexense.com 1 1 DiskBoss Overview DiskBoss is an automated, policy-based data management solution allowing one
What is Deadlock? Two or more entities need a resource to make progress, but will never get that resource. Examples from everyday life:
Chapter 10 Deadlock What is Deadlock? Two or more entities need a resource to make progress, but will never get that resource Examples from everyday life: Gridlock of cars in a city Class scheduling: Two
SharePoint 2010 Technical Case Study: Microsoft SharePoint Server 2010 Social Environment
SharePoint 2010 Technical Case Study: Microsoft SharePoint Server 2010 Social Environment This document is provided as-is. Information and views expressed in this document, including URL and other Internet
EI 338: Computer Systems Engineering (Operating Systems & Computer Architecture)
EI 338: Computer Systems Engineering (Operating Systems & Computer Architecture) Dept. of Computer Science & Engineering Chentao Wu wuct@cs.sjtu.edu.cn Download lectures ftp://public.sjtu.edu.cn User:
Deadlocks were detected. Deadlocks were detected in the DB2 interval statistics data.
Rule DB2-311: Deadlocks were detected Finding: Deadlocks were detected in the DB2 interval statistics data. Impact: This finding can have a MEDIUM IMPACT, or HIGH IMPACT on the performance of the DB2 subsystem.
OpenMP and more Deadlock 2/16/18
OpenMP and more Deadlock 2/16/18 Administrivia HW due Tuesday Cache simulator (direct-mapped and FIFO) Steps to using threads for parallelism Move code for thread into a function Create a struct to hold
Database Management and Tuning
Database Management and Tuning Concurrency Tuning Johann Gamper Free University of Bozen-Bolzano Faculty of Computer Science IDSE Unit 8 May 10, 2012 Acknowledgements: The slides are provided by Nikolaus
Sistemas Operacionais I. Valeria Menezes Bastos
Sistemas Operacionais I Valeria Menezes Bastos Operating Systems: Internals and Design Principles Chapter 1 Computer System Overview Eighth Edition By William Stallings Summary Basic Elements Evolution
CS370: System Architecture & Software [Fall 2014] Dept. Of Computer Science, Colorado State University
![CS370: System Architecture & Software [Fall 2014] Dept. Of Computer Science, Colorado State University](/thumbs/89/99591049.jpg "CS370: System Architecture & Software [Fall 2014] Dept. Of Computer Science, Colorado State University")
Frequently asked questions from the previous class survey CS 370: SYSTEM ARCHITECTURE & SOFTWARE [FILE SYSTEMS] Interpretation of metdata from different file systems Error Correction on hard disks? Shrideep
CSE 120 Principles of Operating Systems
CSE 120 Principles of Operating Systems Spring 2018 Lecture 15: Multicore Geoffrey M. Voelker Multicore Operating Systems We have generally discussed operating systems concepts independent of the number
The Google File System
The Google File System Sanjay Ghemawat, Howard Gobioff, Shun-Tak Leung ACM SIGOPS 2003 {Google Research} Vaibhav Bajpai NDS Seminar 2011 Looking Back time Classics Sun NFS (1985) CMU Andrew FS (1988) Fault
Multiprocessor Systems. Chapter 8, 8.1
Multiprocessor Systems Chapter 8, 8.1 1 Learning Outcomes An understanding of the structure and limits of multiprocessor hardware. An appreciation of approaches to operating system support for multiprocessor
CS 33. Files Part 2. CS33 Intro to Computer Systems XXI 1 Copyright 2018 Thomas W. Doeppner. All rights reserved.
CS 33 Files Part 2 CS33 Intro to Computer Systems XXI 1 Copyright 2018 Thomas W. Doeppner. All rights reserved. Directories unix etc home pro dev passwd motd twd unix... slide1 slide2 CS33 Intro to Computer
Chapter 10: File System. Operating System Concepts 9 th Edition
Chapter 10: File System Silberschatz, Galvin and Gagne 2013 Chapter 10: File System File Concept Access Methods Disk and Directory Structure File-System Mounting File Sharing Protection 10.2 Silberschatz,
Lecture 23 Database System Architectures
CMSC 461, Database Management Systems Spring 2018 Lecture 23 Database System Architectures These slides are based on Database System Concepts 6 th edition book (whereas some quotes and figures are used
File Systems. Chapter 11, 13 OSPP
File Systems Chapter 11, 13 OSPP What is a File? What is a Directory? Goals of File System Performance Controlled Sharing Convenience: naming Reliability File System Workload File sizes Are most files
Programming Guide Guest SDK 3.5
Guest SDK 3.5 Revision: 20071129 Item: SDK-ENG-Q407-126 You can find the most up-to-date technical documentation on our Web site at http://www.vmware.com/support/ The VMware Web site also provides the
DB2 Performance Essentials
DB2 Performance Essentials Philip K. Gunning Certified Advanced DB2 Expert Consultant, Lecturer, Author DISCLAIMER This material references numerous hardware and software products by their trade names.
CSC Operating Systems Fall Lecture - II OS Structures. Tevfik Ko!ar. Louisiana State University. August 27 th, 2009.
CSC 4103 - Operating Systems Fall 2009 Lecture - II OS Structures Tevfik Ko!ar Louisiana State University August 27 th, 2009 1 Announcements TA Changed. New TA: Praveenkumar Kondikoppa Email: pkondi1@lsu.edu
Helix Versioning Engine Administrator Guide: Fundamentals
Helix Versioning Engine Administrator Guide: Fundamentals 2017.2 October 2017 Copyright 1999-2018 Perforce Software. All rights reserved. Perforce Software and documentation is available from www.perforce.com.
MASSACHUSETTS INSTITUTE OF TECHNOLOGY Computer Systems Engineering: Spring Quiz I Solutions
Department of Electrical Engineering and Computer Science MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.033 Computer Systems Engineering: Spring 2011 Quiz I Solutions There are 10 questions and 12 pages in this
Announcements. Computer System Organization. Roadmap. Major OS Components. Processes. Tevfik Ko!ar. CSC Operating Systems Fall 2009
CSC 4103 - Operating Systems Fall 2009 Lecture - II OS Structures Tevfik Ko!ar TA Changed. New TA: Praveenkumar Kondikoppa Email: pkondi1@lsu.edu Announcements All of you should be now in the class mailing
Utilizing Linux Kernel Components in K42 K42 Team modified October 2001
K42 Team modified October 2001 This paper discusses how K42 uses Linux-kernel components to support a wide range of hardware, a full-featured TCP/IP stack and Linux file-systems. An examination of the
AST: scalable synchronization Supervisors guide 2002
AST: scalable synchronization Supervisors guide 00 tim.harris@cl.cam.ac.uk These are some notes about the topics that I intended the questions to draw on. Do let me know if you find the questions unclear
P4Merge User Guide October 2017
P4Merge User Guide 2017.3 October 2017 Copyright 1999-2018 Perforce Software. All rights reserved. Perforce Software and documentation is available from www.perforce.com. You can download and use Perforce
Chapter 8 Virtual Memory
Operating Systems: Internals and Design Principles Chapter 8 Virtual Memory Seventh Edition William Stallings Operating Systems: Internals and Design Principles You re gonna need a bigger boat. Steven
BMC Configuration Management (Marimba) Best Practices and Troubleshooting. Andy Santosa Senior Technical Support Analyst
BMC Configuration Management (Marimba) Best Practices and Troubleshooting Andy Santosa Senior Technical Support Analyst 9/3/2006 Agenda CM Infrastructure CM Inventory CM Subscription CM Software Distribution
Last Class: Synchronization Problems!
Last Class: Synchronization Problems! Reader Writer Multiple readers, single writer In practice, use read-write locks Dining Philosophers Need to hold multiple resources to perform task Lecture 11, page