MSSQL Tips: Tip #27 – SQL Server Master Data Services (MDS) Database Restore Steps

In some cases when you try to restore a Master Dats Services database you can get errors. To complete the restore you may need to perform additional steps.

Please read my latest MSSQL Tip: “SQL Server Master Data Services (MDS) Database Restore Steps“.

Check out all of my tips here: http://www.mssqltips.com/sqlserverauthor/94/svetlana-golovko/.

Backup-Copy-Restore with one script using SQLCMD mode option in SSMS

One of the most common DBAs tasks is databases backup and restore. We do this to test the backups, to refresh test databases, to migrate the databases etc. For some of the tasks we have scripts, for others we need to create scripts on the fly.

Here I would like to share with you a simple script that you will need to run in SQL Server Management Studio’s (SSMS) SQLCMD mode. This script will allow you to create the backup, copy the backup file and restore it on the different server – all from a single SSMS window.

Start new query in SSMS and under “Query” menu select “SQLCMD Mode”:

3-7-2013_1

Run this script (replace host name and SQL Server Instance name for the source and destination variables):

Script

Note, that I commented out the “MOVE” option. Remember to add it to the script if your destination database files’ locations are different from the source.

Get the script from the SkyDrive here.

Does compression enable checksum for the backups automatically?

I am not sure if this is a bug or an issue with BOL documentation, but BOL says here http://msdn.microsoft.com/en-us/library/ms186865.aspx: “(CHECKSUM) Specifies that the backup operation will verify each page for checksum and torn page, if enabled and available, and generate a checksum for the entire backup. This is the default behavior for a compressed backup”.

So, let’s run some tests.

 

Create three backups: backup a database without compression, then backup a database with compression and finally backup it with CHECKSUM:

BACKUP DATABASE msdb TO DISK = ‘K:\BACKUPS\msdb_compressed.bak’

WITH COMPRESSION, COPY_ONLY

BACKUP DATABASE msdb TO DISK = ‘K:\BACKUPS\msdb_NOT_compressed.bak’

WITH COPY_ONLY

BACKUP DATABASE msdb TO DISK = ‘K:\BACKUPS\msdb_CHECKSUM.bak’

WITH CHECKSUM, COPY_ONLY

Run RESTORE HEADERONLY for the backups created without CHECKSUM:

RESTORE HEADERONLY FROM DISK = ‘K:\BACKUPS\msdb_compressed.bak’

RESTORE HEADERONLY FROM DISK = ‘K:\BACKUPS\msdb_NOT_compressed.bak’

Actual results: Both results have “HasBackupChecksums” column equal to 0.

Expected Results: Expecting “HasBackupChecksums” column to have value equal to 1 for the compressed backup (based on documentation).

 

Run RESTORE HEADERONLY for the backup created with CHECKSUM:

 

RESTORE HEADERONLY FROM DISK = ‘K:\BACKUPS\msdb_CHECKSUM.bak’

“HasBackupChecksums” column is equal to 1 as expected.

It does not look that CHECKSUM enabled automatically when COMPRESSION is enabled for the backup.

 

So, the questions are:

  • Is this the documentation issue?
  • Does compression really automatically enables checksum for the backups?

I have sent these questions to Microsoft, so if you want to know the answers as well – come back later. I will publish thier response as soon as I get it.

Backup Default SQL Server Trace files

Default Trace first has been introduced in SQL Server 2005 and could be really useful for Database Administrators in troubleshooting.

There is really great article about what Default Trace is and what you can do with it if you want to learn more:

http://www.simple-talk.com/sql/performance/the-default-trace-in-sql-server—the-power-of-performance-and-security-auditing/

Today I would like to share with you my solution on how to set up an automated trace files’ backups.

How many times you thought: “If I set this audit last month I would know who changed the server configuration”. Unfortunately sometimes we are too busy with our day to day work or do not think that we will ever need to check who changed that specific option on SQL Server and when it was changed. One of the reasons could be that nobody has access to the SQL Server except you. Or you know what you do and documented every one of your steps. But what about those new applications that require sysadmin privileges on SQL Server to install applications and databases? You ask vendor what are the requirements and their response is: “SQL Server 2005 or higher”. And then you discover that installation process actually changed your SQL Server configuration to enable CLR.

This is not real life example, but very close. It took me a couple of hours to go through the tape backups to find out when I changed the server configuration, because I could not remember changing it at all! So, it WAS an application installation.

The first problem when you go through the tape backups is that it is very time consuming. The tape could be offsite; you need to involve backup guys. The second problem is that the trace file we are looking for may not even be on the tape, especially if you backup trace files only once a day. The number of trace files is limited to five files and the new file is generated when file’s size reaches 20 MB. At the same time the oldest file is deleted.

There are lots of solutions out there to have specific records imported to a database from the trace file. One of the latest articles by Aaron Bertrand could be found here:

http://www.mssqltips.com/sqlservertip/2695/maintaining-sql-server-default-trace-historical-events-for-analysis-and-reporting/?utm_source=dailynewsletter&utm_medium=email&utm_content=headline&utm_campaign=201257

But this does not quite work for me. I do not know what I may need next time, so capturing just specific records do not really work for me. Considering that I use DDL triggers for audit and that I need to check old trace files probably once a year, I do not want to import all records to the database, because this is another database to support and this includes backups, maintenance etc.

To save myself time when I needed to check something the next time I came up with solution to backup trace files. This solution will allowed me to make sure that ALL trace files are on the tape backup as well as allow me to will keep an archive of the trace files on the server for the quick access. The solution has been designed and tested for SQL Server 2008 R2.

We want to have all archived Default Trace files on the same server, but in different folder. The files that are archived would also be compressed (zipped).

A little challenge here is that we cannot backup the latest trace file because it will be still in use. We can will only backup “closed” traces.

Components/Requirements:

  1. Archive folder on the server (make sure that SQL Server account or account that will be used to run the archiving job has rights to create and delete files in this folder). To make the scripting more generic I have created the folder under the “Log” folder (location of the Default Trace on your SQL Server instance).
  2. WMI SQL Server Alert.
  3. SQL Server Job to respond to alert (modify job to backup files).
  4. SQL Server Agent “Replace tokens for all job responses to alerts” option is enabled.
  5. Database mail set up.
  6. Operator exists to get test e-mails.
  7. Gzip utility to “zip” archived files.

Note: 5 and 6 only required for the testing. You may skip testing part as this is just to demonstrate files rollover process and response.

Implementation Steps (including testing):

  • This solution requires SQL Server Agent option set to “Replace tokens for all job responses to alerts” which may not be desirable if you are running a very secure environment.

EXEC msdb.dbo.sp_set_sqlagent_properties@alert_replace_runtime_tokens=1

OR set it up using SQL Server Management Studio through the SQL Server Agent Properties:

Note that if SYSTEM account has been removed from logins or SYSADMIN server role you may have issues with WMI alerts.

  • First we need to find location of the Default Trace on your SQL Server instance:
SELECT LEFT(path, LEN(path)-CHARINDEX('\', REVERSE(path)))  FROM sys.traces WHERE id = 1

  • Now – create subfolder for the archived trace files (it could be any other folder that you want to backup to the tapes):

  • Create job to test/confirm that WMI event (trace file rollover) triggers e-mail and that “Replace tokens” option has been configured properly and works. You will need to replace @notify_email_operator_name parameter’s value and @recipients e-mail in the script below. Also replace @profile_name with your DB Mail profile. Remember, that we are just testing WMI now. Do not test this in your production environment!
USE [msdb]
GO
 BEGIN TRANSACTION
DECLARE @ReturnCode INT
SELECT @ReturnCode = 0
 IF NOT EXISTS (SELECT name FROM msdb.dbo.syscategories WHERE name=N'DBA' AND category_class=1)
BEGIN
EXEC @ReturnCode = msdb.dbo.sp_add_category @class=N'JOB', @type=N'LOCAL', @name=N'DBA'
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
 END
 DECLARE @jobId BINARY(16)
EXEC @ReturnCode =  msdb.dbo.sp_add_job @job_name=N'WMI Response - Default Trace File Rollover',
            @enabled=1,
            @notify_level_eventlog=0,
            @notify_level_email=2,
            @notify_level_netsend=0,
            @notify_level_page=0,
            @delete_level=0,
            @description=N'Sends notifications to DBA when Default Trace File Rollover event occures',
            @category_name=N'DBA',
            @owner_login_name=N'sa',
            @notify_email_operator_name=N'DBA_Operator', @job_id = @jobId OUTPUT
-- replace N'DBA_Operator' with operator you created
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
 EXEC @ReturnCode = msdb.dbo.sp_add_jobstep @job_id=@jobId, @step_name=N'Sends e-mail in response to alert',
            @step_id=1,
            @cmdexec_success_code=0,
            @on_success_action=1,
            @on_success_step_id=0,
            @on_fail_action=2,
            @on_fail_step_id=0,
            @retry_attempts=0,
            @retry_interval=0,
            @os_run_priority=0, @subsystem=N'TSQL',
            @command=N'EXEC msdb.dbo.sp_send_dbmail
    @profile_name = ''DBServerAlerts'',
    @recipients = ''firstName.LastName@maildomain.com'',
    @body =   ''File Name: $(ESCAPE_SQUOTE(WMI(FileName)));
                        Start Time: $(ESCAPE_SQUOTE(WMI(StartTime)));
                        Server Name: $(ESCAPE_SQUOTE(WMI(ServerName)));
                        Instance Name: $(ESCAPE_SQUOTE(WMI(SQLInstance)))'',
    @subject = ''Default Trace File Rollover event - $(ESCAPE_SQUOTE(WMI(SQLInstance)))'' ;'
,
-- replace firstName.LastName@maildomain.com with e-mail you will be using for the testing;
-- replace @profile_name with your DB Mail profile
            @database_name=N'master',
            @flags=4
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
EXEC @ReturnCode = msdb.dbo.sp_update_job @job_id = @jobId, @start_step_id = 1
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
EXEC @ReturnCode = msdb.dbo.sp_add_jobserver @job_id = @jobId, @server_name = N'(local)'
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
COMMIT TRANSACTION
GOTO EndSave
QuitWithRollback:
    IF (@@TRANCOUNT > 0) ROLLBACK TRANSACTION
EndSave:
 GO
  • Create alert:
EXEC msdb.dbo.sp_add_alert @name=N'WMI - Default Trace File Rollover',
   @message_id=0,
   @severity=0,
   @enabled=1,
   @delay_between_responses=0,
   @include_event_description_in=1,
   @category_name=N'[Uncategorized]',
   @wmi_namespace=N'\\.\root\Microsoft\SqlServer\ServerEvents\MSSQLSERVER',
   @wmi_query=N'select *  from TRACE_FILE_CLOSE',
   @job_name=N'WMI Response - Default Trace File Rollover' -- name of the job created in step 4
GO

Note that @wmi_namespace will be different for the named instance (example: \\.\root\Microsoft\SqlServer\ServerEvents\INSTANCENAME).

  • Initial testing.

It may take some time to get response from this alert when you test this. This will depend on how busy your system is (how often trace files reach 20 MB size). It may take actually several hours on your test SQL Server. To speed up trace files generation I run this script when I do my testing (it runs about 2 minutes on my test server):

CREATE TABLE tempdb.dbo.ThisIsLongNameToFillOutTraceFile
       (
       ThisIsLongNameToFillOutTraceFile_c1 uniqueidentifier NOT NULL  CONSTRAINT column_a_un1 UNIQUE,
       ThisIsLongNameToFillOutTraceFile_c2 uniqueidentifier NOT NULL  CONSTRAINT column_a_un2 UNIQUE,
       ThisIsLongNameToFillOutTraceFile_c3 uniqueidentifier NOT NULL  CONSTRAINT column_a_un3 UNIQUE,
       ThisIsLongNameToFillOutTraceFile_c4 uniqueidentifier NOT NULL  CONSTRAINT column_a_un4 UNIQUE,
       ThisIsLongNameToFillOutTraceFile_c11 uniqueidentifier NOT NULL  CONSTRAINT column_a_un5 UNIQUE,
       ThisIsLongNameToFillOutTraceFile_c12 uniqueidentifier NOT NULL  CONSTRAINT column_a_un6 UNIQUE,
       ThisIsLongNameToFillOutTraceFile_c13 uniqueidentifier NOT NULL  CONSTRAINT column_a_un7 UNIQUE,
       ThisIsLongNameToFillOutTraceFile_c14 uniqueidentifier NOT NULL  CONSTRAINT column_a_un8 UNIQUE,
       ThisIsLongNameToFillOutTraceFile_c111 uniqueidentifier NOT NULL  CONSTRAINT column_a_un9 UNIQUE,
       ThisIsLongNameToFillOutTraceFile_c112 uniqueidentifier NOT NULL  CONSTRAINT column_a_un10 UNIQUE,
       ThisIsLongNameToFillOutTraceFile_c113 uniqueidentifier NOT NULL  CONSTRAINT column_a_un11 UNIQUE,
       ThisIsLongNameToFillOutTraceFile_c114 uniqueidentifier NOT NULL  CONSTRAINT column_a_un12 UNIQUE,
       ThisIsLongNameToFillOutTraceFile_c211 uniqueidentifier NOT NULL  CONSTRAINT column_a_un13 UNIQUE,
       ThisIsLongNameToFillOutTraceFile_c212 uniqueidentifier NOT NULL  CONSTRAINT column_a_un14 UNIQUE,
       ThisIsLongNameToFillOutTraceFile_c213 uniqueidentifier NOT NULL  CONSTRAINT column_a_un15 UNIQUE,
       ThisIsLongNameToFillOutTraceFile_c214 uniqueidentifier NOT NULL  CONSTRAINT column_a_un16 UNIQUE
       )  ON [PRIMARY]
DROP TABLE dbo.ThisIsLongNameToFillOutTraceFile
GO 3000

Here is example of e-mail that you will get when trace file rollover will occur:

You will see on the server the new file (log_2088.trc) created at the same timetime:

  • Now, when we validated that WMI alerts work and you have server configured properly we can replace the job with the final version:

There will be two steps – to copy closed trace file to the archive folder and to zip the copied file:

Step 1:

Command (in one line):

for /f "tokens=1,2,3,4,5 delims=\ " %a in ("$(ESCAPE_SQUOTE(WMI(FileName)))") do 
xcopy $(ESCAPE_SQUOTE(WMI(FileName))) %a\%b\%c\%d\defaultTraceArchive

Step 2:

Command (in one line):

for /f "tokens=1,2,3,4,5 delims=\ " %a in ("$(ESCAPE_SQUOTE(WMI(FileName)))") do 
cmd /c "%a\%b\%c\%d\defaultTraceArchive\gzip.exe -9 %a\%b\%c\%d\defaultTraceArchive\*.trc"

 

Now we have everything ready for the automated backup of the Default Trace files.

 

Final script:

DBRefresh_BackupDefaultTraceFiles.sql