Saturday, January 30, 2016

DBCC IND and DBCC PAGE

DBCC IND: It is one of the undocumented commands, It is not supported by Microsoft. This command helps in identifying the page numbers that table or index is using. The syntax looks like below. 

Syntax: 
DBCC IND ('DBName' or DBID, 'TableName' or ObjectId, NOn Clustred Index ID)

The third parameter can be the Non Clustered Index Id from Sys.indexes table or 1 or 0 or -1 or -2.  -1 provides complete information about all type of pages ( in row data, row over flow data, IAM, all indexes ) associated with the table.

The result set of DBCC IND looks like below: 




My table using the page numbers 290, 290, 292 and 293. 

Explanation of the what does each column mean: 
PageFID -- the file ID of the page
PagePID -- the page number in the file
IAMFID -- the file ID of the IAM page that maps this page (this will be NULL for IAM pages themselves as they’re not self-referential)
IAMPID -- the page number in the file of the IAM page that maps this page
ObjectID -- the ID of the object this page is part of
IndexID -- the ID of the index this page is part of
PartitionNumber -- the partition number (as defined by the partitioning scheme for the index) of the partition this page is part of
PartitionID -- the internal ID of the partition this page is part of
iam_chain_type -- see IAM chains and allocation units in SQL Server 2005
PageType -- the page type. Some common ones are:
                  1 – data page  2 – index page 3 and 4 – text pages 8 – GAM page 9 – SGAM page 
                  10 – IAM    page 11 – PFS page
IndexLevel -- what level the page is at in the index (if at all). Remember that index levels go from 0 at the   leaf to N at the root page (except in clustered indexes in SQL Server 2000 and 7.0 – where there’s a 0 at the leaf level (data pages) and a 0 at the next level up (first level of index pages))
NextPageFID and NextPagePID -- the page ID of the next page in the doubly-linked list of pages at this level of the index
PrevPageFID and PrevPagePID -- the page ID of the previous page in the doubly-linked list of pages at this   level of the index

DBCC PAGE: It is another undocumented command. As the name suggests this command helps view the contents of the data and index pages. The result of this command is little hard to understand. The syntax looks like below. 

Syntax
DBCC PAGE('DBName' or DBID, FileNumber, PageNumber, PrintOption)
                 PrintOption can be 0 or 1 or 2 or 3 - each option provides different information.

To see the result of this command, lets create a test database, table and insert few records as below. 

USE MASTER
CREATE DATABASE Test
GO

USE Test
CREATE TABLE Pagetable
(
SNo INT IDENTITY(1,1),
Name VARCHAR(50) NOT NULL,
Gender varchar(5),
Fathername varchar(50),
DOB DATETIME NOT NULL,
CONSTRAINT pk_pagetable PRIMARY KEY (Sno),
CONSTRAINT uq_name_father UNIQUE (Name, Fathername)
)
GO

INSERT INTO DBO.Pagetable (Name,Gender,Fathername,DOB) 
VALUES('Tim','M','Tom','1999-03-15') 

INSERT INTO DBO.Pagetable (Name,Gender,Fathername,DOB) 
VALUES('Jim','M','Cam','1999-03-15') 

INSERT INTO DBO.Pagetable (Name,Gender,Fathername,DOB) 

VALUES('Tina','F','Mik','1999-03-15') 

Now lets use the DBCC PAGE command to see the connects of this Test database. But we dont know which page does this database is using. So first run the below DBCC IND command, it results as below screenshot. 

DBCC IND ('test',pagetable,-1)

Result:











Let’s say I am interested in viewing the DBCC PAGE contents of the PageID 286, run the below command.
And we need to turn on the trace 3604 to view the results.

DBCC TRACEON (3604)
DBCC PAGE ('test',1,286,3) with TABLERESULTS

Result:



Tuesday, January 26, 2016

Recovering Master Database

Reasons that force you to recover (Rebuild and\or Restore) the master database:
As we all know master database is very important system database. The master database that records the existence of all other databases and the location of those database files and records the initialization information for SQL Server, SQL server wont even start without master database. Master database contains instance-wide metadata like logon accounts, linked servers, endpoints and system configuration settings.

1) Deletion of any important logins, linked servers or any other SQL object that gets stored in master database.
2) Damaged and unusable master database may be because of disk failure or software failure. 

In 
case 1 we can get back to normal state by restoring the master database from a latest backup copy.
In case 2 we need to rebuild the master database which is nothing but recreating a whole new SQL Server and then restore the system databases and user databases using good latest backup copies.

Restoring the master database from a backup copy:
Let's take I am with case 1. I have my SQL Server looks like something like below with some test logins\Linked servers and always on setup:




I have taken a backup of my master DB and now I am deleting all my link servers and logins to see how I can recover them from the backup that I took. My SQL Server looks like below after deleting those objects:


Step 1: We need to open SQL Server Configuration Manager and right click on SQL Server service to edit the properties to add Startup Parameter -m then click Add, Apply and OK and restart the SQL Server services, this will start the SQL Server in single user mode.  We can't restore master database when SQL Server running is not running in single user mode. 


Step 2: Now open the Command Prompt and run SQLCMD.exe with switches -S (SQL Server Name \Instance Name), -U (Login name) -P (password), depending on where you are running it from. When I did my demo I was RDP'd to the box so I didn't gave switches. And it opens a connection to the SQL Server, enter the T-SQL to restore master database as below. You will finally see a success message as underlined. 

Remember only Sysadmin account can connect in Single User mode. If you have trouble executing SQLCMD.exe from command prompt, you can browse to the location E:\Microsoft SQL Server\110\Tools\Binn where you can see SQLCMD.exe.


Step 3: As final step “go to SQL Server Configuration Manager and remove the Startup Parameter -m and restart the SQL Server service”, that's it now we should able to see all the deleted linked servers\logins. 


Rebuilding the master database:
Let’s take I am with case 2. My master database is so badly corrupted which not even allowing the SQL Server service to start. For demo purpose I have managed to corrupt my master database log file (mastlog.ldf). My healthy SQL Server looks something like below. I have taken the backups of all my system databases, as I need to restore system databases after rebuild them. 




And I have stopped my SQL Server services and I have corrupted my Master database ldf file. And tried to start my SQL Server services as expected it didn't start with the below error.



And looked at the Windows Event Viewer application logs, I found below errors which clearly points to the master database corruption. 



Even I tried to start SQL Server in minimal configuration mode with startup parameter (-f) as a last resort that didn't work either. Now we are left with only one option rebuilding master database. 

Rebuilding process:
I have my Setup Bootstrap in C drive at below location, you will see the Setup.exe inside the Setup bootstrap folder under the SQLServer2014 folder (this folder name changes by version). Open the command prompt as administrator, browse to the folder and run the setup.exe with following the parameters. Replace Admin_Account\Admin_Account_Password with account\Password that has sa rights, InstanceName with MSSQLSERVER for default instance or Name of the instance for named instance. 

Setup /QUIET /ACTION=REBUILDDATABASE /INSTANCENAME="InstanceName" /SQLSYSADMINACCOUNTS="Admin_Account" /SAPWD= "Admin_Account_Password"


In my case the rebuilding process has last for around 5 mins, and you can find the detailed summary of the rebuilding process with final result in summary.txt in the log folder under Setup bootstrap folder. After rebuilding process my instance looks something like below with no logins, User databases. Even the system databases are just empty shells.  


Now it’s time to restore the master, msdb and model databases from the backups. As I said earlier I have backed up all my system databases right before corrupting my master database. First start with restoring the master database use the case Restoring the master database from a backup copy from the above tip in this same post. Then restore msdb and model using my previous post from here.  And my user databases got recovered as soon as my master database is restored from backup, as the user Databases information already exists in my master database.

And let’s say you don't have a master database backup copy and you have corrupted master database. Before you start fixing it try to save the msdb and model db mdfs and ldfs to a safe location. Now try rebuild the master database and you need to attach all the user databases manually and create all the logins, linked servers etc. And then stop the SQL Server service and replace the newly built empty msdb and model database files with the files that you saved earlier and restart the SQL Server services, this way at least you can save time on recreating jobs and other msdb and model database related stuff. 



Sunday, January 24, 2016

Recovering the MSDB or Model Databases

Situations\scenarios that force you to recover the msdb:
1) You have a production server with lot of jobs\alerts\operators\notifications on it. By mistake you have deleted bunch of jobs\alerts\Notifications and you have no idea what they are. 
2) Non repairable database corruptions on msdb.

And you will be on safe side if you have a recent proper backup of msdb. 

Restoring msdb with a recent backup copy:
In Scenario 1 with Valid Recent Backup: Restoring msdb is very similar to restoring any other user database. You can do it through GUI or Script. And similar to any other user DB you need to gain exclusive access on database. Better stop SQL Server agent which always open 1 or 2 connections to msdb.
                 

In Scenario 1 with more or less equivalent msdb Backup copy from different server:
If you have a more or less equivalent msdb on stage server which you want use to restore damaged (corrupted) production msdb to minimize the msdb data loss and time to recreate all the msdb contents from scratch on production. Restoring process in this scenario is very similar but we have to make sure stage and production SQL server's Versions and Builds are matching. We can only restore a msdb backup copy from different server if both target and source SQL Server Versions and Builds matches. Below is the error that we get if they don't match. 
Msg 3168, Level 16, State 1, Line 2
The backup of the system database on the device C:\msdb.bak 
cannot be restored because it was created by a different version 
of the server (10.50.1600) than this server (10.50.2500).
Msg 3013, Level 16, State 1, Line 2
RESTORE DATABASE is terminating abnormally.

In Scenario 2 with Valid Recent Backup: When the msdb is in suspect mode and non recoverable, GUI doesn't work to restore. We have to  use the restore script to restore the backup. 

Error that we get when we try GUI for restoring a corrupted msdb. 

Below is the restore script to use:

Use [Master] 
GO
Restore database msdb from disk=N'E:\E_MSSQL\MSSQL12.MSSQLSERVER\MSSQL\Backup\msdb.bak' with file=1, Nounload, replace, stats=5
GO

Rebuilding msdb using instmsdb sql script or Templats:
In Scenario 2 with No Latest Backup: In this case be prepared for all the data loss. We only can get the brand new msdb into functioning state by rebuilding it using the instmsdb script. Below are the steps. 
  1. Stop the SQL server services.
  2. Start the SQL Server services from the command line by running the NET START MSSQLSERVER /T3608. (Trace Flag 3608 Prevents SQL Server from automatically starting and recovering any database except the master database.)
  3. Then detach the msdb by running the command SQLCMD ‐E ‐S ‐dmaster ‐Q"EXEC sp_detach_db msdb" . and rename the existing msdb files to something else. 
  4. Now run the instmsdb.sql script to rebuild the msdb, SQLCMD ‐E ‐S <SQLSERVERNAME> i"E:\E_MSSQL\MSSQL12.MSSQLSERVER\MSSQL\ Install\instmsdb.sql" ‐o"E:\E_MSSQL\MSSQL12.MSSQLSERVER\MSSQL\Install\instmsdb.o ut", then review the instmsdb.out file for any errors.
  5. Then Stop and start the SQL Server in normal mode with no trace flags.  using  NET STOP MSSQLSERVER and NET START MSSQLSERVER
  6. Reapply any SPs\CUs that are on the SQL Server instance.
  7. And create all the lost jobs\alerts\operators\notifications\schedules etc.
OR 
Using the msdb database templates to rebuild the msdb. 
  1. Simply stop the SQL Server services. 
  2. Copy msdb files from  location E:\E_MSSQL\MSSQL12.MSSQLSERVER\MSSQL\Binn\Templates and overwrite the damaged msdb files at E:\E_MSSQL\MSSQL12.MSSQLSERVER\MSSQL\DATA,  and restart the SQL Server services.  This will give the same result as rebuilding msdb using instmsdb script.  (Location may vary depending on the installation style)
***** In Case of model database *****: The above mentioned procedures pretty much works with the model database as well. There will be very minmul data loss with model.  If I don't have a latest model database backup, I prefer to use files from templates and later make any necessary changes to database. 

Thing to learn: Make sure your backup methods are covering system databases as well. If necessary take Transactions log backups for system DBs also depending on how frequently your master and msdb will be modified. My suggestion is at least take a daily Full backup of all system DBs.         

Tuesday, January 12, 2016

SQL Server Checkpoints

Concept Of Checkpoint: 
Database engine performs the modifications to databases in memory also know on buffer cache and does not write these pages to disk after every change. To harden the changes to disk SQL Server issues periodic Checkpoints on each database. The checkpoint process is an automatic process which runs at specified intervals by SQL Server. Checkpoints will be issued by database. The Checkpoints will be given manually. 

Why Checkpoint:
A checkpoint creates a known good point from which the SQL Server Database Engine can start applying changes contained in the log during recovery after an unexpected shutdown or crash.

When a checkpoint issued on a database no matter how it has occurred the same set of things will happen. Important things happen when a Checkpoint is issued: 
1) The dirty pages (modified pages) will be hardened to physical Disk. 
2) And it writes the log records from log buffer to physical file. 
3) The LSN of the Checkpoint will be recorded in to the database boot page in the dbi_checkptLSN field. 

The Database Engine supports four types of checkpoints: 

1) Automatic Checkpoint: This will be Issued automatically in the background to meet the upper time limit suggested by the recovery interval server configuration option. The default recovery-interval value is 0, which allows the Database Engine to automatically configure the recovery interval. Typically, the default recovery interval results in automatic checkpoints occurring approximately once a minute for active databases and a recovery time of less than one minute. Higher values indicate the approximate maximum recovery time, in minutes. For example, setting the recovery interval to 3 indicates a maximum recovery time of approximately three minutes.


2) Indirect Checkpoint: This will be Issued in the background to meet a user-specified target recovery time for a given database. The default is 0, which indicates that the database will use automatic checkpoints, whose frequency depends on the recovery interval setting of the server instance. Setting the target recovery time to greater than 0 causes the database to use the indirect-checkpoints and establishes an upper-bound on recovery time for that particular database. We can find this option on database properties --> in Options tab --> in Recovery section.  



3) Manual Checkpoints: Issued when you execute a Transact-SQL CHECKPOINT command. The manual checkpoint occurs in the current database for your connection. By default, manual checkpoints run to completion. Optionally, the checkpoint_duration parameter specifies a requested amount of time, in seconds, for the checkpoint to complete.
Command for manual Checkpoint: 
CHECKPOINT [ checkpoint_duration ]

4) Internal checkpoints: These type of checkpoints issued by various server operations. Below is the list:
    > Internal checkpoint will be issued during graceful shutdown of server.
    > It will be issued when teh database recovery model changed from Full\Bulk-logged to Simple. 
    > it will issued during the database backup. 
    > It will be issued on DBs with simple recovery model when the T-log is 70% full. 
    > Alter Database command to add or remove a data\log file also initiates a checkpoint
    > Checkpoint also takes place when the recovery model of the DB is Bulk-Logged and a minimally logged
       operation is performed.
    > It will be performed when a DB Snapshot created. 

Factors Affecting the Duration of Checkpoint Operations:
In general, the amount time required for a checkpoint operation increases with the number of dirty pages that the operation must write. By default, to minimize the performance impact on other applications, SQL Server adjusts the frequency of writes that a checkpoint operation performs. Decreasing the write frequency increases the time the checkpoint operation requires to complete. SQL Server uses this strategy for a manual checkpoint unless a checkpoint_duration value is specified in the CHECKPOINT command.

The performance impact of using checkpoint_duration depends on the number of dirty pages, the activity on the system, and the actual duration specified. For example, if the checkpoint would normally complete in 120 seconds, specifying a checkpoint_duration of 45 seconds causes SQL Server to devote more resources to the checkpoint than would be assigned by default. In contrast, specifying a checkpoint_duration of 180 seconds would cause SQL Server to assign fewer resources than would be assigned by default. In general, a short checkpoint_duration will increase the resources devoted to the checkpoint, while a long checkpoint_duration will reduce the resources devoted to the checkpoint. SQL Server always completes a checkpoint if possible, and the CHECKPOINT statement returns immediately when a checkpoint completes. Therefore, in some cases, a checkpoint may complete sooner than the specified duration or may run longer than the specified duration.

I have gone through Paul Randal's article at www.sqlskills.com and two other sources as below.
Here are the links to the source articles for more details:
http://www.sqlskills.com/blogs/paul/how-do-checkpoints-work-and-what-gets-logged/
https://msdn.microsoft.com/en-us/library/ms189573.aspx
http://www.sarabpreet.com/2013/08/checkpoint/746