Get Started with SQL Server containers with cloned databases

Transcription

1 Get Started with SQL Server containers with cloned databases SQL Server containers with database cloning supports delivery of large, writable SQL Server environments in seconds, with minimal storage requirements. A companion article describes the use of smaller databases run in the container s private file system, see Getting Started with SQL Server containers and in-container databases. SQL Server clones are based on Windows Virtual Hard Drives (VHD), and support creation of differencing disks which are writable cloned database environments. The VHD Is created with a Dockerfile (a plain text configuration file), using Full and Differential backups, or raw database files, and SQL Server scripts. The VHD is a full byte copy of the data, and is included in the SQL Server image. Containers created from the image include cloned database files that can be used by conventional SQL Server instances. The process is supported on the Windocks host, or on a network Windows Server file share, as shown below. Refer to for details on Windocks use with network file shares. Database clones are created in seconds, occupy <40 MB, and are writable. Storage requirement for each clone expands dynamically as changes are made. Clones share read-only access to the parent VHD, and are well suited for Development and Test use. Clones are not recommended for performance and database stress testing. SQL Server images are built with a configuration file (Dockerfile) that includes one or more SETUPCLONING commands that are executed on the Windocks host, or attached file share. SQL Server scripts can be applied during the image build. The example is included in the \windocks\samples folder, and uses a SQL Server 2012 backup located in the dbbackups folder.

2 All files and sub-folders in the same directory as the Dockerfile are copied to the host, where specified files are COPIED into the container file system. As a result, it is important to locate database files or backups in a *separate* directory from the Dockerfile, to avoid copying the DB or backup files unnecessarily. In the example the Full backup is located on the host. For network attached file shares a universal path would be used. Each file is specified with a full path to each file, and striped backups are supported. Each database occupies a separate line, and scores of databases can be included in the same image. Instructions for setting up a network attached file share are available on the windocks.com site (see Set up a Network File Share for Windocks ). Start the Windocks Service and web UI Windocks is installed as a Windows Service, and will start automatically following a system reboot. The Windocks client is also installed on the system path, so docker commands can be used on a command prompt window. See the Windocks Command Line Reference for details, and use of Docker clients on remote systems. Windocks includes a web based User Interface. Open a Firefox or Chrome browser on the Windocks host, and direct the browser to localhost. Once the Windocks web UI resolves enter the local loopback address: The Windocks web UI is now visible, and displays the images available. From a remote workstation, using Firefox or Chrome enter the URL of the Windocks host IP address. Ensure the Windocks host firewall is configured to allow inbound traffic to ports to (container ports), 2375 for the Windocks daemon, and SQL Server default port of Use the Windocks web UI to build a custom SQL Server image To build a SQL Server image click the choose files tool and navigate to the Dockerfile and scripts (\windocks\samples\testfastclonefromfullbackup). Highlight the files, right-click and select, enter an image name and build. The Dockerfile and scripts are copied to the Windocks host for execution. The SETUPCLONING commands are executed on the host, and create the VHD on the host or on a network attached file share (as defined by the Dockerfile).

3 Multiple databases can be moved or restored, and striped backups are supported. The VHD is built in the same folder as the backup, or the first disk defined when using striped backup files that span multiple disks. Large databases that require VHDs that span multiple disks are also supported. Building the image takes time normally associated with restoring the backup, as the image is a full byte copy of the data. When the image is available enter a name for a new container, and click on create. The web page will auto refresh, but the connect button can also be used to refresh the image list.. New containers and database clones are delivered in approximately 30 seconds, irrespective of size of the VHD, and each clone occupies less than 40 MB. The container includes an assigned port, SQL sa

4 password, database name, and full path to the cloned databases. At this point the user can work with the environment by starting the container. The path to the database files are presented, and can be used by non-container users with conventional SQL Server instances. Support for sharing these files is part of the server configuration, set in the node file located in \windocks\config directory: CLONE_USERS_PERMITTED=" domain\username, domain\username2." The cloned databases reflect the steps involved in the image build, including any data masking scripts applied. Starting the container attaches the cloned databases, and makes the container accessible. Remote clients use the host IP address (port separated by a comma), and the SQL sa credentials to access.

5 Using the Web UI to create updated SQL Server images: Images that include cloned databases are updated with Differential backups and scripts, through another image build. As before, the build is defined by a Dockerfile. In this case we use a Differential backup that is included in the \windocks\dbbackups folder. The Dockerfile used is located at \windocks\samples\testfastclonefromdifferentialbackup. The update is applied to the parent image (FROM newsql in our example), and applies a differential backup with the SETUPCLONING DIFF command. SQL Server scripts are copied into the container and run during the build. Any files that need to be copied are located in the same folder as the Dockerfile. Note: Windocks image names are case sensitive! Updated images based on backups depends on the Full backup parent image, and becomes a second full byte copy of the environment. Updated images can be deleted and replaced, but require the original parent image. Windocks prevents images from being deleted while containers are present that depend on them.

6 To build the updated image, select the choose files tool and navigate to the Dockerfile and script, highlight both files, right-click and select. Assign a name to the image, and click on create. Once the image is available, assign a new container name and create. The page will refresh and display the container and access details. Using the Windocks Command Line Interface Windocks is an independent port of Docker s open source to Windows, and supports standard Docker clients. The Windocks installation includes a Docker.exe that can be copied to any Windows client, and can be used locally on the Windocks host (location is \windocks\client). Docker clients for mac and linux machines are also available, see Additional Resources at the end of this article for more information. The docker client on the Windocks host is included in the system path, so is available via a standard command prompt window or PowerShell on any directory path. For client machines either setup the system path, or navigate to the directory where the Docker.exe is located to work with the following commands. The Windocks CLI supports a standard set of Docker commands. When working from a remote client, the syntax takes on an added complexity, and requires the host firewall to be configured to allow inbound traffic on ports , the default daemon port of 2375, and SQL Server port >docker -H=tcp://ip.address.of.host:2375 images Remote client call for >docker images Docker commands used on the Windocks host use the simpler syntax shown below: >docker build t <imagename> <directory> Builds an image only. This command supports SQL Server images with cloned databases.

7 >docker build <directory> Builds an image and container, with the image named with the container name. Does not support building an image with cloned databases. >docker commit <option> <instruction><containerid> <imagename> Commits a container to create a new image. Does not support creating images with cloned databases. >docker create <option> <image> >docker exec <containerid> <command> >docker images >docker ps Creates a new stopped container. Executes a command in a container. Lists available images on the host. Lists containers on the host. >docker rm <containerid> Removes the containers. Use 2-3 digits, sufficient to achieve a unique match to the full container id. >docker rmi <imagename> >docker run d <option> <image> >docker start <containerid> Removes the image. Use the full image name. Creates a running container. Starts the container >docker stop <containerid> Stops the container Containers created with >docker create or run d can include assigned ports, names, and SQL sa passwords. A commit can include a Dockerfile instruction to ADD, COPY, or RUN, with the --change option. -e SA_PASSWORD= Pa55word## -p <port> --name <containername> --change instruction file --cidfile=<path to folder>

8 DockerFiles: Dockerfiles are plain text configuration files that support the creation of new containers and images. A number of examples are included in the \windocks\samples directory. The Docker client will copy all files and folders that are in the Dockerfile location to the host. It is important to include only files that are desired in the container or image to be located with the Dockerfile. Supported Dockerfile commands include: FROM<image> ADD <file> ADDDB <dbname> <mdf> <ndf> <ldf> COPY <file> ENV <environment variable> note: SA_PASSWORD is not current supported. EXPOSE <port> RUN <file> SETUPCLONING FULL <dbname> <path to Full backup> SETUPCLONING DIFF <dbname> <path to Differential backup> SETUPCCLONING RAW <dbname> <path to DB files> Note, SETUPCLONING instructions must be used with the >docker build t command. Database files referenced by the ADDDB should be located in the same directory as the Dockerfile. Using the Command Line Interface Below the docker client is run locally on the Windocks host. The >docker images command confirms the daemon is running, and the SQL Server 2014 image is available. An image assigning the image name of clone (>docker build -t clone). When building images (using the >docker build t) the container built is automatically deleted, and the return string is disregarded. A second >docker images command confirms the new clone image is available.

9 Containers with the clone environment are delivered with a >docker run d <image> command. In the example we assign a port at 10050, a container name of pauls, and the return string includes the containerid, with the port and SQL sa credentials. The container is accessible via SQL Management Studio and other tools, for remote access use the SQL sa credentials, or credentials for user logins setup in the default instance. For local access use the loopback address with a comma separating the port (ie., ,1000X). For remote access use the host IP address, with a comma separator. To update the image, build a new image. We can apply a SQL Server change script to the Full backup image, and create a modified version of the current image. Or, we can apply a Differential backup with or without scripts. In the example below, note that the image refers to the parent VHD (clone), that we are updating the image with the differential backup, and running the same data masking scripts used earlier.

10 The build uses the SETUPCLONING DIFF command. Again, the container that results from the build t is deleted and the return string is disregarded. The newclone image is confirmed with a >docker images command. Pros and Cons for SQL Server containers with Cloned databases The processes outlined in this article are ideal for delivery of Dev and Test environments of any size. Each container is delivered in approximately 30 seconds, irrespective of the size of the image, is writable, and only occupies <40 MB on delivery. A team can work with scores of identical environments on a single shared server, and save hours each week in VM maintenance (and on costs of VM infrastructure). Containers are created quickly, so this workflow is ideal for short-lived instances needed for Dev and Test. The web UI provides Developers an easy to use self-service environment. An additional advantage is that the cloned databases can be used with conventional SQL Server instances, or the containers, so Windocks is both a container engine as well as general purpose SQL Server cloning solution. Cloned environments are not well suited for performance or stress testing, and system planning can be challenging. As changes are made to the clone, the disk footprint of the clone will expand dynamically. Additional Resources and Notes: 1) For work with larger and more complex database environments, see the companion article on Getting Started with SQL Server containers with in container data 2) Understanding and Using Windocks 2.2 3) Installing and Configuring Windocks 2.2 4) Windocks 2.2 Command Line Reference 5) Windocks containers operate with DNS:

11 6) To understand Windocks licensing options for organizations: 7) Forthcoming Jenkins CI pipeline support? 8) For information on working with multi-tier environments, including.net see: 9) For technical support 10) For information on how Windocks compares to Microsoft s new containers in Windows Server 2016: