Parallel pg_dump. Parallel pg_dump. Maxing out hardware resources for database copy, backup and restore. Joachim Wieland. CHAR(10), 2nd July 2010

Transcription

1 Parallel pg_dump Maxing out hardware resources for database copy, backup and restore Joachim Wieland CHAR(10), 2nd July 2010

2 The Idea Parallel pg_dump Current pg_dump does not use too much of the available hardware pg_dump runs with one process on the frontend over one connection using one backend process current servers have almost all multiple CPUs (and/or Hyperthreading) We can probably do better.

3 How does pg_dump work (current single-threaded behavior) Connect to the database Read the schema data, create internal catalog of dumpable objects (=TOC) Determine the dump order (dependencies) Write a first version of the TOC to the dump file Dump all objects in the calculated order Update the TOC with the byte position of each object

4 How does pg_dump work (current single-threaded behavior) Connect to the database Read the schema data, create internal catalog of dumpable objects (=TOC) Determine the dump order (dependencies) Write a first version of the TOC to the dump file Dump all objects in the calculated order Update the TOC with the byte position of each object

5 How does pg_dump work (current single-threaded behavior) Connect to the database Read the schema data, create internal catalog of dumpable objects (=TOC) Determine the dump order (dependencies) Write a first version of the TOC to the dump file Dump all objects in the calculated order Update the TOC with the byte position of each object

6 How does pg_dump work (current single-threaded behavior) Connect to the database Read the schema data, create internal catalog of dumpable objects (=TOC) Determine the dump order (dependencies) Write a first version of the TOC to the dump file Dump all objects in the calculated order Update the TOC with the byte position of each object

7 How does pg_dump work (current single-threaded behavior) Connect to the database Read the schema data, create internal catalog of dumpable objects (=TOC) Determine the dump order (dependencies) Write a first version of the TOC to the dump file Dump all objects in the calculated order Update the TOC with the byte position of each object

8 How does pg_dump work (current single-threaded behavior) Connect to the database Read the schema data, create internal catalog of dumpable objects (=TOC) Determine the dump order (dependencies) Write a first version of the TOC to the dump file Dump all objects in the calculated order Update the TOC with the byte position of each object

9 How does pg_dump work

10 How a parallel pg_dump could work (future multi-threaded behavior) Connect to the database with multiple processes In the master process, read the schema data Determine the dump order (dependencies) Wait for a worker process to become free and assign a new object to it Wait for all worker processes to finish Write the TOC file

11 How a parallel pg_dump could work (future multi-threaded behavior) Connect to the database with multiple processes In the master process, read the schema data Determine the dump order (dependencies) Wait for a worker process to become free and assign a new object to it Wait for all worker processes to finish Write the TOC file

12 How a parallel pg_dump could work (future multi-threaded behavior) Connect to the database with multiple processes In the master process, read the schema data Determine the dump order (dependencies) Wait for a worker process to become free and assign a new object to it Wait for all worker processes to finish Write the TOC file

13 How a parallel pg_dump could work (future multi-threaded behavior) Connect to the database with multiple processes In the master process, read the schema data Determine the dump order (dependencies) Wait for a worker process to become free and assign a new object to it Wait for all worker processes to finish Write the TOC file

14 How a parallel pg_dump could work (future multi-threaded behavior) Connect to the database with multiple processes In the master process, read the schema data Determine the dump order (dependencies) Wait for a worker process to become free and assign a new object to it Wait for all worker processes to finish Write the TOC file

15 How a parallel pg_dump could work (future multi-threaded behavior) Connect to the database with multiple processes In the master process, read the schema data Determine the dump order (dependencies) Wait for a worker process to become free and assign a new object to it Wait for all worker processes to finish Write the TOC file

16 How a parallel pg_dump could work $ pg_dump -j 3 <dbname>

17 What we have already We have parallel pg_restore already (by Andrew Dunstan for PostgreSQL 8.4) How does it work? remember: TOC entries contain dependency information remember: TOC entries contain the byte position of each object in the dump file as soon as all dependencies of one object are fulfilled, the object itself can be restored pg_restore forks off up to n processes a restoring process can directly seek to that position

18 Why we do not yet have a parallel pg_dump Several pg_dump processes need to share the same snapshot Dumping with several processes to a single file is probably a bad idea

19 Snapshot sharing A snapshot defines which version of the database a client program (its transaction actually) sees. For a consistent backup, all pg_dump processes need to see the same version of the database. I don t have a solution for that currently. 2 different use cases for pg_dump: regular nightly backup (writers active) database migration (all writing applications stopped)

20 New archive format "directory" Main reason: Scalability Other reasons: A directory dump could span several drives Managing a large dump file is hard (several GB/TB) How it could look like $ ls 1.dat 3.dat 5.dat 2.dat 4.dat TOC But there are also some disadvantages...

21 New archive format "directory" Something looks wrong here #1 $ ls 1.dat 3.dat 5.dat 2.dat TOC

22 New archive format "directory" Something looks wrong here #1 $ ls 1.dat 3.dat 5.dat 2.dat TOC a file got lost

23 New archive format "directory" Something looks wrong here #3 $ ls -l 1.dat :32 2.dat :32 3.dat :32 4.dat :18 TOC :32

24 New archive format "directory" Something looks wrong here #3 $ ls -l 1.dat :32 2.dat :32 3.dat :32 4.dat :18 TOC :32 a file got overwritten

25 New archive format "directory" Something looks wrong here #4 $ ls -l backup-yesterday 1.dat :32 2.dat :32 3.dat :32 TOC :32 $ ls -l backup-today 1.dat :18 2.dat :18 3.dat :18 TOC :18 $ ls -l backup-today 1.dat :18 2.dat :18 3.dat :32 TOC :18

26 New archive format "directory" Something looks wrong here #4 $ ls -l backup-yesterday 1.dat :32 2.dat :32 3.dat :32 TOC :32 $ ls -l backup-today 1.dat :18 2.dat :18 3.dat :18 TOC :18 $ ls -l backup-today 1.dat :18 2.dat :18 3.dat :32 TOC :18 Two backups got mixed

27 New archive format "directory" Backup set in different folders $ ls backup-part1/ 1.dat 3.dat 5.dat $ ls backup-part2/ 2.dat 4.dat TOC

28 New archive format "directory" Checks that we should do: Check if all required files are there Make sure no unrelated file is there Make sure they are all from the same backup optional: Make sure they have the correct size optional: Make sure they have a matching checksum

29 Dump order Parallel pg_dump Keep processes busy Ideally all should start and finish at the same time

30 Dump order Parallel pg_dump Keep processes busy Ideally all should start and finish at the same time

31 Live Demo Parallel pg_dump Live demo

32 Some statistics Parallel pg_dump

33 Table sizes Parallel pg_dump

34 Further work Parallel pg_dump Parallel copy (parallel version of pg_dump psql):

35 Further work Parallel pg_dump Wouldn t it be cool if we could dump one table with multiple processes? We might want to run several pg_dump client processes on different hosts Check and evaluate other compression libraries (lzf) to improve data throughput further

36 Further work Parallel pg_dump Wouldn t it be cool if we could dump one table with multiple processes? We might want to run several pg_dump client processes on different hosts Check and evaluate other compression libraries (lzf) to improve data throughput further

37 Further work Parallel pg_dump Wouldn t it be cool if we could dump one table with multiple processes? We might want to run several pg_dump client processes on different hosts Check and evaluate other compression libraries (lzf) to improve data throughput further

38 Some statistics (lzf)

39 Thank you Parallel pg_dump Questions?