Week In Review – June 15th 2012

It has been a busy week at SteveStedman.com. Here is a quick summary of this weeks postings:

I think my favorite for the week was the Seven Hundredth download of the Database Health Reports project. This project is really starting to take off. I am glad that people are using it and getting value out of it.

A successful week 2 of Common Table Expression month at SteveStedman.com

Introduction to Recursive CTEs

Day 7 of Common Table Expression Month (June) at SteveStedman.com, today we will be taking a look at the introduction to recursive CTEs.

These queries will be using that database that was set up in a previous posting on the CTE_DEMO Sample Database, if you haven’t set up the sample database, download it and set it up now.

The recursive feature of CTEs is perhaps one of the most powerful things you can do with a Common Table Expression

What is Recursion

Recursion is a concept in programming where a function calls itself. Many of the typical C programming interview questions around data structures like trees, linked lists, and other structures often times use recursion. For instance in the following T-SQL example, the scalar function Fibonacci calculates the Fibonacci sequence using recursion, by calling itself.  This is accomplished by calling the function name inside of the body of the function.


CREATE FUNCTION dbo.Fibonacci (@Num integer, @prev integer, @next integer)
RETURNS VARCHAR(4000)
AS
BEGIN
DECLARE @returnValue as VARCHAR (4000) = cast(@prev as varchar(4000));
IF (@Num > 0)
BEGIN
IF (LEN(@returnValue) > 0)
BEGIN
SET @returnValue = @returnValue + ',';
END
SET @returnValue = @returnValue + dbo.Fibonacci(@Num - 1, @next, @next + @prev) ;
END

RETURN @returnValue;
END
GO

To call the function you simply include in a select statement, with the first parameter being the number of Fibonacci numbers to calculate, and the second and third parameters are always 0 and 1. The second and third parameters are set to 0 and 1 to prime the recursive function, and are used internally to pass the recursive call the current and previous values.

select dbo.Fibonacci(10, 0, 1);

Which produces the following output using SSMS:
FIbonacci_Scalar

Recursion With a CTE

Doing recursion as shown above with a function could be considered the brute force way of doing it. The common table expression way of doing recursion is much more elegant, and you can use it even if you don’t have the database permissions needed to create stored procs or functions.

Here is an example of a recursive CTE that queries a heirarchy of store departments:


use [cte_demo];
-- Recursive CTE
;WITH DepartmentCTE(id, Department, Parent, Level) AS
( SELECT id, department, parent, 0 as Level
 FROM Departments
 WHERE parent is NULL
 UNION ALL -- and now for the recursive part
 SELECT d.id, d.department, d.parent,
 DepartmentCTE.Level + 1 as Level
 FROM Departments d
 INNER JOIN DepartmentCTE
 ON DepartmentCTE.id = d.parent)
SELECT *
 FROM DepartmentCTE
 ORDER BY Parent;

The first time I looked at a recursive CTE, I was just confused by the syntax. Lets walk through the different parts and it may make more sense, to start with I will add some white space to the query to split it up a bit:

RecursiveCTE1

Anchor Query

The anchor query is the part of the CTE that starts the recursion. This is the part of the query that is run first that the recursive part will build from.

RecursiveCTE2

Recursive Query

The UNION ALL keyword is used to separate the anchor query from the recursive part.

RecursiveCTE3

After the UNION ALL comes the Recursive Query

RecursiveCTE4

What makes the recursive part of the CTE query recursive is the way that it references itself

Executing the Recursive CTE

Once the recursive CTE has been written, it can be executed just like any other CTE, SELECT everything FROM the CTE name, in this case we are also using ORDER BY to sort the results.

RecursiveCTE5

With the CTE complete, lets take a look at the results.

RecursiveCTE6

Now lets break the results up into what was generated by the anchor and what was generated by the recursive query.

RecursiveCTE7

Whats Next

Now that you understand recursive CTE’s we can get into some really interesting recursive options over the rest of CTE month at SteveStedman.com. Stick around something new most every day.

Related Links:

Common Table Expressions Book

If you enjoyed this posting, and want to learn more about common table expressions, please take a look at my book on CTE’s at Amazon.com. The book is titled Common Table Expressions - Joes 2 Pros® - A CTE Tutorial on Performance, Stored Procedures, Recursion, Nesting and the use of Multiple CTEs.

250th Blog Post at SteveStedman.com

This is my 250th blog post at Steve Stedman.com.  Many of the post over the last year have been focused on Common Table Expression topics as I finished my CTE Book, however with last month being DBCC Command Month here there were many posts completely unrelated to CTEs.

Another category that accounts for many of the posts is the Database Health Reports project which started out fully contained on the SteveStedman.com website, but late last summer Database Health Reports Project was moved to a site of its own at DatabaseHealth.SteveStedman.com.

My goal with each blog post is to share something of value with the SQL Server community. Very rarely will I post anything that is not directly related to my experiences with Microsoft SQL Server. Over the next year I intend to continue to add more valuable content for the SQL Server community.

Common Table Expression (CTE) Month at SteveStedman.com

This month is going to be Common Table Expressions Month at SteveStedman.com.   My goal is to post one valuable post on a Common Table Expressions topic each day for the entire month.

What is a Common Table Expression

A Common Table Expression (know as CTE) is a type of in memory result set, similar to a view or a derived table, however the CTE only exists for the duration of a single query. CTE’s have many uses from simplifying huge derived table queries, to doing recursive queries that would otherwise be very difficult to accomplish with SQL Server.  Every version of SQL Server from SQL Server 2005 on includes the ability to use CTE’s, including SQL Azure. CTE’s are available in the SQL Server Express, SQL Server Standard Edition, SQL Server Enterprise, SQL Server Developer, SQL Azure, and any other version of Microsoft SQL Server from SQL Server 2005 and on.

If you like this or any of the postings on CTE’s, please take a look at my book on Common Table Expressions. The book is currently available at Amazon.com in paperback and Kindle versions.

So What is a Common Table Expression?

Here is a preview of what a CTE looks like, stick around for tomorrows blog which describes how it all comes together, and what you can do with it.


;WITH MyCommonTableExpression AS
(

SELECT *

FROM Departments

)

SELECT *

FROM MyCommonTableExpression

Stay tuned for tomorrows posting on

Sample Databases

For Common Table Expressions month at SteveStedman.com I will be providing several sample queries most of which can be run in the cte_demo database which is available for download.

My Background With Common Table Expressions

I first learned about CTE’s in 2011, at which point I was just confused at the strange syntax. I had found a query that someone else had provided on their blog which used a CTE for something, I don’t even remember what it was used for. When trying to figure out what that query was doing, I was forced to figure out this strange new WITH syntax that I had not seen before.  From there I became obsessed with understanding everything about Common Table Expressions.

After that introduction to CTE’s I quickly discovered that I was using CTE’s more and more often, and that there were some really cool tricks like recursive queries that you could do with CTEs.

In late 2011 I pitched a session for SQL Saturday#108 in Redmond WA on February 25th 2012 at the Microsoft Campus, and the session was accepted.  This was going to be my first time presenting at a SQL Saturday and I had better learn everything that there is to know about CTE’s so that I don’t look foolish.  I spent the next several months preparing my SQL Saturday session, and researching as much as I possibly could. I presented at the SQL Saturday and it went quite well. After the presentation several people made a few suggestions, and I adapted my presentation for SQL Saturday #114 in Vancouver BC Canada on March 17th 2012.

Next I had the opportunity to present at Seattle Code Camp 2012 on June 16th.  I cleaned up the presentation a bit more, learned a few more tricks with CTE’s and presented at Seattle Code Camp.

Later in 2012 I presented at SQL Saturday 172 in Portland Oregon, and SQL Sat 166 in Olympia Washington. At one of those SQL Saturdays I met Rick Morelan from Joes2Pros and we discussed writing a book. I pitched the idea of  a book on Common Table Expressions and Rick asked the question of “A whole book on Common Table Expressions, really, what would you cover?” So I threw out ideas on recursion, de-duplication of data, tree-path’s, Fibonacci sequence, and data paging. I still don’t think that he was entirely convinced that it would be worth a whole book, so I put together an outline to show what I had in mind. A couple weeks later I was working with Rick and the team at Joes2Pros to write the book on Common Table Expressions. But there aren’t any other books focused only on CTE’s.

The next few months I spent writing the book on CTE’s learning every detail that I possibly could so that I didn’t get anything wrong, and updating and refining the book. I can safely say that I have spent over 1000 hours over the last 2 years researching and understanding Common Table Expressions.

Several months later May 18, 2013 I was presenting at SQL Saturday in Redmond WA again, and I had the opportunity to give the CTE presentation again. But it wasn’t the same presentation for 15 months ago, it had evolved, improved, and covered far more information that I could ever fit into the hour session.  The book was announced for the first time at that SQL Saturday session, and it was now available on Amazon.com.

Related Links:

Common Table Expressions Book

If you enjoyed this posting, and want to learn more about common table expressions, please take a look at my book on CTE’s at Amazon.com

Common Table Expressions Presentations Updated.

I used to be able to do the Common Table Expression in a single 1 hour session, but after writing the book on Common Table Expressions, I discovered that I just can’t fit it into an hour with the detail that Common Table Expressions deserve. I would rather go into depth so that everyone can master CTEs rather than just breezing over the highlights and leaving everyone wondering.

For future SQL Saturdays and other presentations the sessions have been broken out as follows. Currently I have proposed the following 2 sessions for SQL Saturday Denver.

Common Table Expressions – Introduction

Have you ever wanted to create a recursive query, but didn’t see how to do it. With the Common Table Expression session you will learn everything needed to start using CTEs for recursive queries, as temporary views, and to use the result set multiple times in the same query. Learn how simplify query syntax using CTEs. One of the most overlooked features of SQL Server is the CTE which not only simplifies the query, but gives you the ability to do things that would otherwise be impossible (or at least very challenging) with SQL Server. The class is designed for people who haven’t used CTEs before, or for those who want to learn the basics of CTEs including data paging. This session pairs well with the Advanced Common Table Expression session.

Common Table Expressions – Advanced

You have been introduced to Common Table Expression, you understand the ;WITH syntax, but you want to know more. Learn how to recursive queries work with CTEs and how to display hierarchical data. Did you know that you can INSERT, UPDATE and DELETE data from CTEs. Some of the common use cases for CTEs will be covered including finding holes in patterns, finding and removing duplicate data, string parsing, and more.  See how CTEs compare to SQL Server 2012 offset and fetch paging techniques. Get an in depth understanding of the performance behind a common table expression. Understand when the CTE is the right solution, and the wrong solution to use. Finally we will take a look at some classic recursive algorithms and how they can be implemented with CTEs. This session pairs well with the Introduction to Common Table Expression session.

Common Table Expression accepted for SQL Saturday 212 in Redmond WA

My Common Table Expression presentation was accepted for SQL Saturday 212 in Redmond WA.

This is the session that I am the most familiar with, and that I have presented the most times. I think this will be the 8th time I have presented this session, and after every previous presentation it has been improved.

Over the last several months I have been working with the Rick Morelan and the exceptional team at Joes2Pros to write a book on common table expressions.  This book is in the final cleanup stage, and I hope to have the a couple of the first printed copies of the book available at the session as a door prize or give-away.

Some of the things that will be covered in this session that I haven’t covered in the session before will be:

  • CTEs and the DELETE statement… Can I delete data from a CTE?
  • The UPDATE statement on a CTE… Will it update the base tables?
  • Interesting performance implications. What is safe and what is not safe with a CTE.

I will also be covering the usual topics:

If I had a little more time, I might be able to rename the presentation to “Everything you ever wanted to know about CTE’s, but were afraid to ask”.

This should be a fun presentation based on the amount of focus I have had on CTE’s over the last few months.

 

See you at Microsoft in Redmond on May 18th for SQL Saturday!