DBCC FreeProcCache

Being day 12 of the DBCC Command month at SteveStedman.com, today’s featured DBCC Command is DBCC FREEPROCCACHE.

Description:

DBCC FREEPROCCACHE is used for purge all of the parsed query plans out of memory. This is commonly used in development environments, but not as common in a production environment.

Use in a development environment is common, for instance when you are working on performance tuning, or parameterization of queries. You can clear the procedure cache with DBCC FreeProcCache, run the program or web page that may be using the database, then see what is in the procedure cache. This can be useful in finding queries that may need to be parameterized. Another way to use would be to find out what queries are being run by some program. To do this you would start by working with a database that is not being used by others, clearing the procedure cache with DBCC FreeProcCache, then run the program you are trying to figure out, then look at what is in the cache, again this is something that could be done in a development or test environment, but I wouldn’t recommend doing it in production.

Use in a production environment should be rare, this is one of the common things to try when SQL Server is having difficulty. If you are are the point that SQL Server is extremely slow to respond and you have been unable to find the cause, one thing to try is to free the procedure cache with DBCC FreeProcCache and see if that fixes the problem.

DBCC FreeProcCache Syntax:

dbcc freeproccache
[ ( @HANDLE | 'POOL NAME' ) ]
[ WITH NO_INFOMSGS ]

Example:

The following example is from a development envioronment using the AdventureWorks2012 Database.

First we connect to AdventureWorks2012 and see what is in the cache.

USE AdventureWorks2012;
GO

SELECT size_in_bytes, text
 FROM sys.dm_exec_cached_plans
 CROSS APPLY sys.dm_exec_sql_text(plan_handle) AS st;

DBCC_FreeProcCache1
Here we see that there is plenty in the cache. Next we cleare the cache with DBCC FreeProcCache and take another look at what is in the cache.

DBCC FREEPROCCACHE;

SELECT size_in_bytes, text
 FROM sys.dm_exec_cached_plans
 CROSS APPLY sys.dm_exec_sql_text(plan_handle) AS st;

After running DBCC FreeProcCache you can see that there is nothing left in the cache.
DBCC_FreeProcCache2When the very next query is run, it will need to be reparsed rather than using an already parsed query in the cache.  This will take a bit longer than if there was already a parsed plan to run.  Lets run 3 queries, then take a look at the cache.

GO
SELECT FirstName, MiddleName, LastName
 FROM person.Person p
 WHERE FirstName like 'John';

GO
SELECT FirstName, MiddleName, LastName
 FROM person.Person p
 WHERE FirstName like 'Mary';
GO
SELECT FirstName, MiddleName, LastName
 FROM person.Person p
 WHERE FirstName like 'Bill';

Notice the GO Statement between each query. This tells SSMS to run each query as a separate batch. Without the GO statement the 3 queries would have been parsed as a single batch.
DBCC_FreeProcCache3
Here we see the results from the three queries. The first two returned results, and the third one had no rows in the result set. Now lets take a look at the cache

SELECT size_in_bytes, text
 FROM sys.dm_exec_cached_plans
 CROSS APPLY sys.dm_exec_sql_text(plan_handle) AS st;

Below we see a total of 20 items in the cache now. the top item highlighted in blue is the query we used to see what was in the cache, the second block highlighted in red contains the 3 queries from above, and the third rest of them are queries being run by SQL or other supporting queries.  For instance line 13 is the dm_exec_sql_text which is called from the query above that checks the plan.DBCC_FreeProcCache4

If we wanted to filter this down to just the queries we had written you could do it by adding a WHERE text LIKE … clause to the query as shown here.

SELECT size_in_bytes, text
 FROM sys.dm_exec_cached_plans
 CROSS APPLY sys.dm_exec_sql_text(plan_handle) AS st
WHERE text LIKE N'SELECT FirstName%';

Here we see that only the three queries show up, and that each of those three takes up about 40K of memory on SQL Server. DBCC_FreeProcCache5So why are there three copies of the same SELECT statement, this seems a bit wasteful. Indeed it does, for more information see an earlier post called How much Procedure Cache memory does one SQL Statement take up? There are ways to correct this.

Using DBCC FreeProcCache For A Specific Plan Handle

If you wanted to clear just a single plan handle, and not all the plan handles, you could use the optional parameter called @handle.

To get the plan handle, we start by modifying our earlier query to show us what is in the plan cache. You could leave out the WHERE clause on your own system, but I have it here to show us just the three queries in question from above.


SELECT size_in_bytes, text, plan_handle
 FROM sys.dm_exec_cached_plans
 CROSS APPLY sys.dm_exec_sql_text(plan_handle) AS st
WHERE text LIKE N'SELECT FirstName%';

DBCC_FreeProcCache7

Here we see the same three query plans from earlier, with an addition column called plan_handle. To free a single plan handle, we would just copy the numeric plan handle, and add that into the DBCC FreeProcCache query.


DBCC FREEPROCCACHE(0x060007000100FF3310B8DA7D0600000001000000000000000000000000000000000000000000000000000000);

SELECT size_in_bytes, text, plan_handle
 FROM sys.dm_exec_cached_plans
 CROSS APPLY sys.dm_exec_sql_text(plan_handle) AS st
WHERE text LIKE N'SELECT FirstName%';

DBCC_FreeProcCache8

Where we only see 2 of the three original queries in the plan cache.

How big is my Procedure Cache?

You can run the following query to check the size of your procedure cache.


SELECT count(*) AS NumPlans, objtype as Type,
SUM(size_in_bytes)/(1024.0*1024.0) AS size_in_mb
FROM sys.dm_exec_cached_plans
GROUP BY objtype;

Which produces the following results on my test server.

DBCC_FreeProcCache9

Database Health Reports and the Plan Cache

You can also view the plan cache using the Database Health Reports application as shown here.

DBCC_FreeProcCache10

Notes:

For more information see TSQL Wiki DBCC freeproccache.

DBCC Command month at SteveStedman.com is almost as much fun as eating jello.

DBCC CleanTable

Being day eight of the DBCC Command month at SteveStedman.com, today’s featured DBCC Command is DBCC CLEANTABLE.

Many times I have worked on a database that has evolved over 10 or more years of changes from different developers and DBAs. One of the signs I see in databases like this is the waste associated with obsolete columns.  For instance someone had an email address column as a varchar(1024) just to have plenty of space, then someone decided to do some normalization and they created a table just to hold email addresses, and the original table only had a id to represent the email address that was foreign keyed to the new table. In a good case the developer was wise enough to remove the original column, but not always. Assume the case where it wasn’t delete, so we do a check, determine that the old varchar(1024) column is not being used so we drop the column. After dropping the column we notice that the table didn’t get any smaller. Thats OK, maybe we will just use that space that we freed up from the column we dropped over time as the table grows, but NO. The space associated with that column is left around.

So to actually free up that space associated with the dropped columns so that it can be used there are 2 common options.

  1. DBCC CleanTable 
  2. Rebuild the clustered index

The purpose of this post is to demonstrate how DBCC CleanTable can be used to regain wasted space.

Description:

DBCC CleanTable is used for clean up waste left around by deleting variable length columns.

DBCC CLEANTABLE Syntax:

dbcc cleantable
(
    { 'database_name' | database_id | 0 }
    , { 'table_name' | table_id | 'view_name' | view_id }
    [ , batch_size ]
)
    [ WITH NO_INFOMSGS ]

The Third Argument is Batch size: This is the number of rows processed in one transaction. If not specified or if value is 0, it means all rows of table will be in 1 transaction. For larger tables it may make sense to process parts of the table in smaller batches.

Example:

First we will create a table to use for the following example

CREATE TABLE [LotsOfWaste] (
 id int, --would normally be an INT IDENTITY
 firstName VARCHAR (8000),
 MiddleName VARCHAR (8000),
 lastName VARCHAR (8000),
 SocialSecurityNumber VARCHAR (8000),
 FavoriteFood VARCHAR (8000),
 FavoriteColor VARCHAR (8000),
 archived bit default 0
 );

First we will fill up the table with 10, 000 rows using the GO statement to repeat our INSERT statement 10,000 times.

GO

set nocount on;
INSERT INTO [dbo].[LotsOfWaste]
 ([firstName]
 ,[MiddleName]
 ,[lastName]
 ,[SocialSecurityNumber]
 ,[FavoriteFood]
 ,[FavoriteColor]
 ,[archived])
VALUES (
  REPLICATE('1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ', cast(rand() * 10 as int))
 ,REPLICATE('1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ', cast(rand() * 10 as int))
 ,REPLICATE('1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ', cast(rand() * 10 as int))
 ,REPLICATE('1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ', cast(rand() * 10 as int))
 ,REPLICATE('1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ', cast(rand() * 10 as int))
 ,REPLICATE('1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ', cast(rand() * 10 as int))
 ,1)
GO 10000

Now to check on the number of pages allocated for this table, and the percentage used. Below we see that there are 1426 page allocated, and of the memory associated with those 1426 pages, 86.3% of that memory is used to accomodate the 10,000 records.

SELECT alloc_unit_type_desc, page_count,
 avg_page_space_used_in_percent, record_count
FROM sys.dm_db_index_physical_stats( DB_ID('dbcc_corruption'),
 OBJECT_ID(N'LotsOfWaste'),
 NULL,NULL,'Detailed')

DBCC_CleanTable1

Now lets drop 4 columns from the table, and take a look at the pages allocated and the space used.


SELECT alloc_unit_type_desc, page_count,
 avg_page_space_used_in_percent, record_count
FROM sys.dm_db_index_physical_stats( DB_ID('dbcc_corruption'),
 OBJECT_ID(N'LotsOfWaste'),
 NULL,NULL,'Detailed')

ALTER TABLE [dbo].[LotsOfWaste] DROP COLUMN [FavoriteColor];
ALTER TABLE [dbo].[LotsOfWaste] DROP COLUMN [FavoriteFood];
ALTER TABLE [dbo].[LotsOfWaste] DROP COLUMN [SocialSecurityNumber];
ALTER TABLE [dbo].[LotsOfWaste] DROP COLUMN [MiddleName];

SELECT alloc_unit_type_desc, page_count,
 avg_page_space_used_in_percent, record_count
FROM sys.dm_db_index_physical_stats( DB_ID('dbcc_corruption'),
 OBJECT_ID(N'LotsOfWaste'),
 NULL,NULL,'Detailed')

The number of pages, percent used, and record count are exactly the same before and after deleting the columns.  This table is not using any less space now that those 4 large columns were deleted.
DBCC_CleanTable2

So maybe we can just insert more rows and that will take advantage of the space that was being used by the 4 deleted columns.  Perhaps…?


SELECT alloc_unit_type_desc, page_count,
 avg_page_space_used_in_percent, record_count
FROM sys.dm_db_index_physical_stats( DB_ID('dbcc_corruption'),
 OBJECT_ID(N'LotsOfWaste'),
 NULL,NULL,'Detailed')

GO
INSERT INTO [dbo].[LotsOfWaste]
 ([firstName]
 ,[lastName]
 ,[archived])
 VALUES (
  REPLICATE('1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ', cast(rand() * 10 as int))
 ,REPLICATE('1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ', cast(rand() * 10 as int))
 ,1)
GO 100
SELECT alloc_unit_type_desc, page_count,
 avg_page_space_used_in_percent, record_count
FROM sys.dm_db_index_physical_stats( DB_ID('dbcc_corruption'),
 OBJECT_ID(N'LotsOfWaste'),
 NULL,NULL,'Detailed')

No, inserting more rows, even rows that are smaller than the original rows does not make use of the space left behind by the column removal.  In this case inserting 100 rows ended up allocating another 4 data pages.

DBCC_CleanTable3

So where do we go from here, how do we reclaim the space associated with the columns that were deleted? Now we will try DBCC CleanTable since that is the topic of this posting.  Lets see what happens.

SELECT alloc_unit_type_desc, page_count,
 avg_page_space_used_in_percent, record_count
FROM sys.dm_db_index_physical_stats( DB_ID('dbcc_corruption'),
 OBJECT_ID(N'LotsOfWaste'),
 NULL,NULL,'Detailed')

DBCC CleanTable(dbcc_corruption, LotsOfWaste);

SELECT alloc_unit_type_desc, page_count,
 avg_page_space_used_in_percent, record_count
FROM sys.dm_db_index_physical_stats( DB_ID('dbcc_corruption'),
 OBJECT_ID(N'LotsOfWaste'),
 NULL,NULL,'Detailed')

Here we see that the number of pages for the table hasn’t changed, and the record count hasn’t changed, but the average spaced used has changed from 86% to 30%.

DBCC_CleanTable4

There are still the same number of pages, but can we actually use those pages? Lets give it a try. Here we will insert 100 rows, and see if the page count grows as we saw it grow earlier…


SELECT alloc_unit_type_desc, page_count,
 avg_page_space_used_in_percent, record_count
FROM sys.dm_db_index_physical_stats( DB_ID('dbcc_corruption'),
 OBJECT_ID(N'LotsOfWaste'),
 NULL,NULL,'Detailed')

GO
INSERT INTO [dbo].[LotsOfWaste]
 ([firstName]
 ,[lastName]
 ,[archived])
 VALUES (
  REPLICATE('1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ', cast(rand() * 10 as int))
 ,REPLICATE('1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ', cast(rand() * 10 as int))
 ,1)
GO 100
SELECT alloc_unit_type_desc, page_count,
 avg_page_space_used_in_percent, record_count
FROM sys.dm_db_index_physical_stats( DB_ID('dbcc_corruption'),
 OBJECT_ID(N'LotsOfWaste'),
 NULL,NULL,'Detailed')

Here we see that the page count has not increased from the before to the after check, the record count has increased due to the 100 rows inserted, and the average spaced used has changed slightly.

DBCC_CleanTable5

What we did here was insert those last 100 rows into space that was not usable prior to running DBCC CleanTable. By cleaning up a table this way we don’t decrease the overall number of pages, the size of the table, but we do allow for possibly many more rows to be inserted into the space that has been allocated.

Warning:

DBCC CleanTable for large tables may cause the transaction log to grow.

Notes:

DBCC CleanTable is doesn’t work on system tables or temporary tables.

For more information see TSQL Wiki DBCC cleantable.

DBCC Command month at SteveStedman.com is almost as much fun as looking at paint swatches at the home improvement store.

DBCC CheckIdent

Being day six of the DBCC Command month at SteveStedman.com, today’s featured DBCC Command is DBCC CHECKIDENT.

Description:

DBCC CHECKIDENT is used for check on the current value in the identity column for a table.  It also reports on the largest value in that column. DBCC CheckIdent can also be used to update or set the next identity value on a table.

If you call DBCC CheckIdent with just a table name, it will report back information, and it will also update the next identity value if there is a value in the table larger than the current next identity value.  This can be dangerous if you are not expecting this. Use the NORESEED parameter to tell DBCC CheckIdent to not make any changes. You can also use the RESEED parameter with a specific number to set the identity value to a that number.

DBCC CHECKIDENT Syntax:

dbcc checkident
(
    'table_name'
    [ , { NORESEED
        | { RESEED [ , new_reseed_value ] }
    } ]
)
    [ WITH NO_INFOMSGS ]

Example:

The following example….

First we create a table called TestSmallIntIdentity with an id of type SMALLINT and a log time.  Keep in mind that a SMALLINT accommodates values from -32768 to 32767. The default on the identity is to default to zero. If you are not familiar with this, take a look at my Are you wasting half the capacity of IDENTITY article.

Notice the table is created with the primary key being the id column to prevent duplicates.


CREATE TABLE [dbo].[TestSmallIntIdentity](
 [id] [smallint] IDENTITY(1,1) NOT NULL,
 [logTime] [datetime] NOT NULL DEFAULT GetDate(),
 CONSTRAINT [PK_TestSmallIntIdentity] PRIMARY KEY CLUSTERED ([id] ASC)
) ON [PRIMARY];

Once the table has been created we insert a single row, and look at the contents of the table, and the number of rows.

INSERT INTO TestSmallIntIdentity DEFAULT VALUES;

SELECT * FROM TestSmallIntIdentity;
SELECT count(*) FROM TestSmallIntIdentity;

DBCC_checkIdent1
At this point we have 1 row in the table. Now lets just add 1000 more rows into the table. This may take a minute to run depending on your server speed.  Once complete, the SELECT * should return 1001 rows, and the count(*) should show 1001.

SET NOCOUNT ON;
INSERT INTO TestSmallIntIdentity DEFAULT VALUES;
GO 1000

SELECT * FROM TestSmallIntIdentity ORDER BY id DESC;
SELECT count(*) FROM TestSmallIntIdentity;

DBCC_checkIdent2In the above example, we can see that the maximum id value in the table is 1001, and therefore the next identity value would be 1002.

Now we delete everything from the table, insert a single row, take a look at the id of the row, and run DBCC CheckIdent to see what we can about the identity.

DELETE FROM TestSmallIntIdentity;

INSERT INTO TestSmallIntIdentity DEFAULT VALUES;

SELECT * FROM TestSmallIntIdentity;
SELECT count(*) FROM TestSmallIntIdentity;

DBCC CHECKIDENT (TestSmallIntIdentity, NORESEED);



DBCC_checkIdent3

DBCC_checkIdent4
As we speculated above the identity value that was inserted is 1002.

Next we delete everything from the table and check the identity, notice that the current identity value is 1002, and the current column value is ‘NULL’ implying in this case that there are no rows in the table. Since the table is empty it should be safe at this point to set the identity value to anything we want it to be (we will get to that in a minute).

DELETE FROM TestSmallIntIdentity;
DBCC CHECKIDENT (TestSmallIntIdentity, NORESEED);

DBCC_checkIdent5
Next we insert one thousand more rows, and check the identity value, and it will show 2002 as the current value.

SET NOCOUNT ON;
INSERT INTO TestSmallIntIdentity DEFAULT VALUES;
GO 1000

DBCC CHECKIDENT (TestSmallIntIdentity, NORESEED);

DBCC_checkIdent6
So what happens if we delete part of the table.  We know that the rows in the table now have id values of 1002 to 2002, so lets delete those that are greater than 1500. From here we check the identity, and see that the identity value is 2002, and the maximum column value is 1500.

DELETE FROM TestSmallIntIdentity WHERE id > 1500;
DBCC CHECKIDENT (TestSmallIntIdentity, NORESEED);

DBCC_checkIdent7

Knowing that the maximum value is 1500, if we are using trusted foreign keys, and can confirm that there are no other references to values in this table that have been deleted, we can just adjust the identity value to be 1501 as shown below.

DBCC CHECKIDENT (TestSmallIntIdentity, RESEED, 1501);

DBCC_checkIdent8

Check the identity value with DBCC CheckIdent, and we see (shown below) that the current identity value is set to 1501, and the max value in the column is 1500.

DBCC CHECKIDENT (TestSmallIntIdentity, NORESEED);

DBCC_checkIdent9
Next we  insert a row into the TestSmallIntIdentity table, and check the the identity after, showing 1502 as the current value in the table, and the current identity value.

INSERT INTO TestSmallIntIdentity DEFAULT VALUES;
DBCC CHECKIDENT (TestSmallIntIdentity, NORESEED);

DBCC_checkIdent10
Now what happens when the identity is full, meaning it has reached its maximum value for the data type used. We will try this by inserting almost enough rows to ready the 32,767 maximum of a SMALLINT data type.

DELETE FROM TestSmallIntIdentity;
DBCC CHECKIDENT (TestSmallIntIdentity, RESEED, 0);
INSERT INTO TestSmallIntIdentity DEFAULT VALUES;
DBCC CHECKIDENT (TestSmallIntIdentity, NORESEED);
GO
INSERT INTO TestSmallIntIdentity DEFAULT VALUES;
GO 32765 -- Run it 32,765 times

DBCC_checkIdent11
Next we insert one row, and check to see where we are at with the identity. Everything looks good at this point, we can insert one more row.

DBCC CHECKIDENT (TestSmallIntIdentity, NORESEED);

DBCC_checkIdent12
So we insert another with no errors, and check our current value, which shows as 32767, but we know the max value in a SMALLINT is 32767, so it must be full.

INSERT INTO TestSmallIntIdentity DEFAULT VALUES;
DBCC CHECKIDENT (TestSmallIntIdentity, NORESEED);

DBCC_checkIdent13
What happens when we insert one more as follows.

INSERT INTO TestSmallIntIdentity DEFAULT VALUES;
DBCC CHECKIDENT (TestSmallIntIdentity, NORESEED);

DBCC_checkIdent14

The Arithmetic overflow error message is shown:
Msg 8115, Level 16, State 1, Line 1
Arithmetic overflow error converting IDENTITY to data type smallint.
Arithmetic overflow occurred.

What do we do now?

Now we can do one of the following:

  • Stop inserting rows into this table.
  • Change the size of the identity column and everywhere else that is foreign keyed to this ID to be something larger than a SMALLINT.
  • Just reset the identity to be the most negative number possible, to buy us some time.

Lets take a look at resetting the identity to be a negative number.  Keep in mind that this only buys us some time, if we filled up the positive numbers is a SMALLINT, we will eventually use up the negative numbers. Warning, you may run into problems later once this increases to 0, and there are no more numbers left in the SMALLINT range.

Here is how to reset the identity to a new value, in this case -32768, or the most negative number that will fit into a SMALLINT.


DBCC CHECKIDENT (TestSmallIntIdentity, RESEED, -32768);

DBCC_checkIdent15
Notice in the above output, when we set the value to -32,768, there was not output that showed we had set it to -32,768. So let’s check to see what is is, using DBCC CheckIdent.  Be sure to use the NORESEED parameter on CheckIdent.  The NORESEED indicates that the check does not adjust the seed value for the identity.  If this is run in this specific case without the NORESEED parameter, it will automatically adjust the identity value since it is less than the column value. It would set it to 32767, and we would end up in the same place as we were before we tried setting it to -32768.


INSERT INTO TestSmallIntIdentity DEFAULT VALUES;
DBCC CHECKIDENT (TestSmallIntIdentity, NORESEED);

DBCC_checkIdent16

Now we will use up most of the negative identity values from the SMALLINT, to get to the point that the identity value is -2.  Here we know that all of the numbers that can fit into the SMALLINT that are less than or equal to -2 have been used, and we know that all the values greater than 0 have been used. the only numbers left are -1, and 0.

GO
INSERT INTO TestSmallIntIdentity DEFAULT VALUES;
GO 32765 -- Run it 32,765 times

-- now check to see where we are at.
DBCC CHECKIDENT (TestSmallIntIdentity, NORESEED);

DBCC_checkIdent17Next one more insert.


-- insert another and check
INSERT INTO TestSmallIntIdentity DEFAULT VALUES;
DBCC CHECKIDENT (TestSmallIntIdentity, NORESEED);

DBCC_checkIdent18

Now we insert one more row, which uses up 0, and there no more numbers left after this insert.


-- and another
INSERT INTO TestSmallIntIdentity DEFAULT VALUES;
DBCC CHECKIDENT (TestSmallIntIdentity, NORESEED);

DBCC_checkIdent19

Now we try one more insert anyway, and find out that we are violating the PRIMARY KEY constraint with duplicate keys.


-- and another
INSERT INTO TestSmallIntIdentity DEFAULT VALUES;
DBCC CHECKIDENT (TestSmallIntIdentity, NORESEED);

DBCC_checkIdent21

Msg 2627, Level 14, State 1, Line 1
Violation of PRIMARY KEY constraint 'PK_TestSmallIntIdentity'. Cannot insert duplicate key in object 'dbo.TestSmallIntIdentity'. The duplicate key value is (1).
The statement has been terminated.

That’s it, we are now out of numbers for this primary key ID value with a SMALLINT data type. The next thing to try would be a larger data type like INTEGER.
Now lets check the identity value, and see how it looks. If we just run DBCC CheckIdent wihtout the NORESEED value it will reseed the value to 32767 without telling us. So lets run it a second time to check as shown here.


DBCC CHECKIDENT (TestSmallIntIdentity);
-- now check, and see that it has already changed.
DBCC CHECKIDENT (TestSmallIntIdentity, NORESEED);

DBCC_checkIdent22

Now we try the final insert, this time it is trying to insert 32767 + 1 which as shown below causes an arithmetic overflow.

INSERT INTO TestSmallIntIdentity DEFAULT VALUES;
DBCC CHECKIDENT (TestSmallIntIdentity, NORESEED);

DBCC_checkIdent23

Summary

You can use DBCC CheckIdent to see what the current identity value is. You can also use it to optionally set the identity value to something you specify, or end up having it automatically set for you.

 

Related:

DBCC CHECKCONSTRAINTS – Check integrity of table constraints.

Are you wasting half the capacity of IDENTITY?

Notes:

For more information see TSQL Wiki DBCC checkident.

DBCC Command month at SteveStedman.com is almost as much fun as calculating identity values.

Rows and Range, Preceding and Following

SQL Server 2012 adds many new features to Transact SQL (T-SQL).  One of my favorites is the Rows/Range enhancements to the over clause. These enhancements are often times referred to as the windowing functions.

Overview:
ROWS PRECEDING, FOLLOWING, UNBOUNDED, refers to the current row and those before or after based on preceding or following.
RANGE PRECEDING, FOLLOWING, UNBOUNDED, means all values in the current range and those before or after.

An example of a need for rows preceding would be to calculate a 3 year trailing average, which needs to look at the current year and the three previous years in the calculation.

Terminology:
ROWS or RANGE- specifying rows or range.
PRECEDING – get rows before the current one.
FOLLOWING – get rows after the current one.
UNBOUNDED – when used with PRECEDING or FOLLOWING, it returns all before or after.
CURRENT ROW

To start out we need a database to work on, so we will create a tsql2012 database. you can use your own database if you wish.

CREATE DATABASE [tsql2012];
GO
USE [tsql2012];

Next we create a table to use for the over clause enhancments of rows and range preceding and following.

-- Table to be used by Over Clause Rows/Range

CREATE TABLE REVENUE
(
[DepartmentID] int,
[Revenue] int,
[Year] int
);

insert into REVENUE
values (1,10030,1998),(2,20000,1998),(3,40000,1998),
 (1,20000,1999),(2,60000,1999),(3,50000,1999),
 (1,40000,2000),(2,40000,2000),(3,60000,2000),
 (1,30000,2001),(2,30000,2001),(3,70000,2001),
 (1,90000,2002),(2,20000,2002),(3,80000,2002),
 (1,10300,2003),(2,1000,2003), (3,90000,2003),
 (1,10000,2004),(2,10000,2004),(3,10000,2004),
 (1,20000,2005),(2,20000,2005),(3,20000,2005),
 (1,40000,2006),(2,30000,2006),(3,30000,2006),
 (1,70000,2007),(2,40000,2007),(3,40000,2007),
 (1,50000,2008),(2,50000,2008),(3,50000,2008),
 (1,20000,2009),(2,60000,2009),(3,60000,2009),
 (1,30000,2010),(2,70000,2010),(3,70000,2010),
 (1,80000,2011),(2,80000,2011),(3,80000,2011),
 (1,10000,2012),(2,90000,2012),(3,90000,2012);

Take a look at the revenue table to see whats there before we start the demo.


USE [tsql2012];

-- first lets look at the REVENUE table

SELECT *
 FROM Revenue;

rows_range_preceding_following1
Contents of the REVENUE table.

Then a quick review of sum and avg.

-- first simple sum and avg aggregates
SELECT sum(Revenue) as TotalRevenue,
 avg(Revenue) as AverageRevenue,
 count(*) as NumRows
 FROM Revenue;

rows_range_preceding_following2

OVER Clause Pre SQL Server 2012

The OVER clause before SQL Server 2012 is really handy to perform aggregates over a different range than your standard grouping.

--First OVER Clause pre SQL 2012
SELECT *,
 avg(Revenue) OVER (PARTITION by DepartmentID) as AverageDeptRevenue,
 sum(Revenue) OVER (PARTITION by DepartmentID) as TotalDeptRevenue
FROM REVENUE
ORDER BY departmentID, year;

rows_range_preceding_following3

ROWS PRECEDING

Now the new features, ROWS PRECEDING specifies the the aggregate functions in the current partition in the OVER clause will consider the current row, and a specific number of rows before the current row.


--ROWS PRECEDING
-- look at the sum of revenue over a trailing 3 year period
SELECT Year, DepartmentID, Revenue,
 sum(Revenue) OVER (PARTITION by DepartmentID
 ORDER BY [YEAR]
 ROWS BETWEEN 3 PRECEDING AND CURRENT ROW) as CurrentAndPrev3
FROM REVENUE
ORDER BY departmentID, year;

rows_range_preceding_following4

ROWS FOLLOWING

Now rows following… The ROWS FOLLOWING option specifies a specific number of rows in the current partition to use after the current row.

-- ROWS FOLLOWING
SELECT Year, DepartmentID, Revenue,
 sum(Revenue) OVER (PARTITION by DepartmentID
 ORDER BY [YEAR]
 ROWS BETWEEN CURRENT ROW AND 3 FOLLOWING) as CurrentAndNext3
FROM REVENUE
ORDER BY departmentID, year;

rows_range_preceding_following5

Both PRECEDING and FOLLOWING

Using both ROWS PRECEDING and ROWS FOLLOWING allows you to do things like calculate an average including the current year and years both before and after the current year.


--ROWS PRECEDING AND FOLLOWING

SELECT Year, DepartmentID, Revenue,
 sum(Revenue) OVER (PARTITION by DepartmentID
 ORDER BY [YEAR]
 ROWS BETWEEN 1 PRECEDING AND 1 FOLLOWING) as BeforeAndAfter
FROM REVENUE
ORDER BY departmentID, year;

rows_range_preceding_following6

UNBOUNDED

UNBOUNDED PRECEDING tells the windowing function and aggregrates to use the current value, and all values in the partition before the current value.

-- ROWS UNBOUNDED PRECEDING
SELECT Year, DepartmentID, Revenue,
 min(Revenue) OVER (PARTITION by DepartmentID
 ORDER BY [YEAR]
 ROWS UNBOUNDED PRECEDING) as MinRevenueToDate
FROM REVENUE
ORDER BY departmentID, year;

rows_range_preceding_following7

When using UNBOUNDED FOLLOWING as follows, this means the use the current row, and consider all rows after the current row in the current partition.

-- ROWS UNBOUNDED FOLLOWING
-- http://stevestedman.com/?p=1485
SELECT Year, DepartmentID, Revenue,
 min(Revenue) OVER (PARTITION by DepartmentID
 ORDER BY [YEAR]
 ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) as MinRevenueBeyond
FROM REVENUE
ORDER BY departmentID, year;

rows_range_preceding_following8

ROWS vs RANGE

ROWS vs RANGE appears to cause much confusion.

ROWS means the specific row or rows specified, and RANGE refers to those same rows plus any others that have the same matching values.

Here is an example of ROWS and RANGE producing the exact same results because the years and department IDs don’t have any duplicates. In this case the ROWS and RANGE are identical.

-- ROWS vs RANGE UNBOUNDED PRECEDING

SELECT Year, DepartmentID, Revenue,
 sum(Revenue) OVER (PARTITION by DepartmentID
 ORDER BY [YEAR]
 ROWS UNBOUNDED PRECEDING) as RowsCumulative,
 sum(Revenue) OVER (PARTITION by DepartmentID
 ORDER BY [YEAR]
 RANGE UNBOUNDED PRECEDING) as RangeCumulative
FROM REVENUE
WHERE year between 2003 and 2008
ORDER BY departmentID, year;

rows_range_preceding_following9

Now if we consider the same query of ROWS vs RANGE with duplicate values. Here we insert a duplicate set of values for 2005, and the results below show that for the year 2005 there are two rows in the result set, and the RowsCumulative column is different form the RangeCumulative.  The rows counts the specific rows it the aggregate calculation, the range counts the current row, plus the other 2005 rows for that department id.

-- INSERT A DUPLICATE VALUE FOR RANGE UNBOUNDED PRECEEDING

INSERT INTO REVENUE
VALUES (1,10000,2005),(2,20000,2005),(3,30000,2005);

-- same query as above
SELECT Year, DepartmentID, Revenue,
 sum(Revenue) OVER (PARTITION by DepartmentID
 ORDER BY [YEAR]
 ROWS UNBOUNDED PRECEDING) as RowsCumulative,
 sum(Revenue) OVER (PARTITION by DepartmentID
 ORDER BY [YEAR]
 RANGE UNBOUNDED PRECEDING) as RangeCumulative
FROM REVENUE
WHERE year between 2003 and 2008
ORDER BY departmentID, year;

rows_range_preceding_following10

Summary

With all the great new features in SQL Server 2012, the windowing functions, ROWS and RANGE, PRECEDING and FOLLOWING updates to the OVER clause are a great addition.

Using a SEQUENCE object to generate letters.

SQL Server 2012 introduces a new T-SQL object used to generate numbers, a SEQUENCE, similar to an identity column. You can think of a sequence as a take a number object, you just ask it to give you a number, and you get a number.

When creating a SEQUENCE you need to specify the following:

  • Start
  • Increment
  • Min Value, Max Value
  • Cycle / No Cycle – starts of when the Max value is hit or not
  • Cache

The SEQUENCE is all about generating numbers…

The question comes up on using a SEQUENCE to generate letters, to rotate through a SEQUENCE of A to Z, then to start over?

Technically a SEQUENCE can only generate numbers, but here is how you can use it to generate letters.

Here is the sample code to do it.


CREATE SEQUENCE lettersSequence
 AS tinyint
 MINVALUE 65
 MAXVALUE 90
 CYCLE
 INCREMENT BY 1 ;

SELECT char(NEXT VALUE FOR lettersSequence);

SELECT char(NEXT VALUE FOR lettersSequence);
SELECT char(NEXT VALUE FOR lettersSequence);
SELECT char(NEXT VALUE FOR lettersSequence);
SELECT char(NEXT VALUE FOR lettersSequence);
SELECT char(NEXT VALUE FOR lettersSequence);
SELECT char(NEXT VALUE FOR lettersSequence);

Enjoy.

Shrinking a Log File on a Test or Development Server

On my computer I run SQL Server 2012, and I use a number of test or development databases. I don’t run backups on these because I don’t care if they get destroyed since they are easy to replace. Recently I upgraded to a SSD, and now my C:\ drive which I am running SQL Server from is really fast, and SQL Server in the development environment is also very fast. The only drawback is the size of the SSD, instead of a 2TB disk, I am now limited to 120GB of fast disk, and I have a second disk with 2 TB.  I ended up with the 120GB drive because I was more focused on performance rather than the overall size.

Yes, this disk is really fast, but suddenly I am need to be more aware of what is using up space because of the limited size. I use the Adventure works for a lot of test scripts, and queries that I might be blogging about, and I noticed that my AdventureWorks log file was 1.5GB or about 1% of my entire disk space, and that is about all waste, on this particular server.

My first thought was to just move the databases over to my slower non-SSD drive, but I want things as fast as possible, so that is not an option.

WARNING: This advice is intended for a development or test database where it doesn’t matter if the data file is lost.  This could be done in production once it was practiced in a test or development environment.  Just be sure you know what you are doing before running it on a production system.

First lets take a look at the size of the AdventureWorks log file.

AdventureWorksLogSize

Its been a while since I set up this database, and I don’t remember how it was originally configured for backups, but first lets set the Recovery Model to be simple, meaning that we don’t need transaction log backups. The Recovery model can be set from the Database Properties dialog on the Options page.

Simple

Next we can check to see how much space is being used, and how much space is available in the AdventureWorks2012_log file. To do that we just run the following query in the AdventureWorks database.

USE AdventureWorks2012;
GO
SELECT (size - CAST(FILEPROPERTY(name, 'SpaceUsed') AS int))/128.0 AS AvailableMB,
 name
FROM sys.database_files;

Which when run produces the following output:
waste_space

Showing that in the log file there is 1.5GB of available space that could be reclaimed. Now on to reclaim it. To reclaim the space we are going to use DBCC SHRINKFILE on the log. When running this be sure to run it only on the log. If DBCC SHRINKFILE is run against the data file, it usually ends up fragmenting all of the indexes and decreasing performance while the space is reclaimed.


USE AdventureWorks2012;
GO
DBCC SHRINKFILE (AdventureWorks2012_Log);
GO

Producing the following output:

shrinkfile_results

After the shrink the size is 63 data pages, 56 used. To see the actual numbers in MB, just run the original query from above to see the following results:

nowast_space

From here we see that the available space on the log file has been cleared up, and if we look at the file sizes on disk, we see the following:

filesystem_after_shrink

Rather than 1.5GB the file is now 504KB or about half a megabyte.

Overall the plan behind this was to free up space on my SSD, and that is what we have done here by shrinking the log file.