Louis's profileThe SQL Doctor is In (Re...PhotosBlogListsMore Tools Help

Blog


    May 07

    Using application locks to implement a critical section in T-SQL code

    This is being added to the addendum (located here) for my "Pro SQL Server 2005 Database Design and Optimization" book and would have appeared in the pessimistic locking section on page 478 in Chapter 9: Coding for Integrity and Concurrency; Pessimistic Locking. You can download the entire addendum here: Addendum.

    The problem of the critical section is a very common problem.  Very often it is troublesome for more than one connection to have access to a given section of code.  For example, when you need to fetch a value, increment it, and keep it unique amongst other callers that could be executing the code simultaneously.  Exclusively locking the part of the table works, but it can be troublesome when either:

    • The code is being executed within another transaction that can end up keeping more data locked than actually necessary
    • Only one minor section needs to be single threaded through, but allow simultaneous access otherwise.
    • The speed in which the data is accessed is so fast that you are likely to fetch the same data within microseconds of each other, leading to duplicates or deadlocks
    • When the single threading is not for table access.  You may want to write to a file of some sort, or use some other resource that is not table based.

    A technique to leave tables unlocked but single thread access of some sort manually, we can use an application lock to lock a section of code. 

    Note that the applock must be used/honored manually in every piece of code where this matters so there is a loss of safety associated with this process.  If there is any concern with what other processes might do, be sure to still assign proper concurrency and locking hints to the code.

    To demonstrate a very common problem of building a unique value without using identities (for example, if you have to create an account number with special formatting/processing,) I have created the following table:

    create table applock
    (
        applockId int primary key,  --the value that we will be generating with the procedure
        connectionId int,                --holds the spid of the connection so you can who creates the row
        insertTime datetime default (getdate()) --the time the row was created, so you can see the
                                                                 --progression
    )

    Next, a procedure that starts an applock, fetches some data from the table, increments the value, and stores it in a variable.  I have a delay parameter so you can tune up the problems by making the delay between increment and insert more pronounced.  There is a parameter to turn the applock on and off also, as that will help you test to see it work, and not work without the applock.

    create procedure applock$test
    (
        @connectionId int,
        @useApplockFlag bit = 1,
        @stepDelay varchar(10) = '00:00:00'
    ) as
    set nocount on
    begin try
        begin transaction
            if @useApplockFlag = 1 --turns on and off the applock for testing
                begin
                    exec @retval = sp_getapplock @Resource = 'applock$test', @LockMode = 'exclusive';
                    if @retval < 0
                        begin
                            declare @errorMessage nvarchar(200)
                            set @errorMessage = case @retval
                                                                when -1 then 'Applock request timed out.'
                                                                when -2 then 'Applock request canceled.'
                                                                when -3 then 'Applock involved in deadlock'
                                                                else 'Parameter validation or other call error.'
                                                            end
                            raiserror (@errorMessage,16,1)
                        end
                end

        --get the next primary key value
        declare @applockId int   
        set @applockId = coalesce((select max(applockId) from applock),0) + 1

        --delay for parameterized amount of time to slow down operations
        --and guarantee concurrency problems
        waitfor delay @stepDelay

        --insert the next value
        insert into applock(applockId, connectionId)
        values (@applockId, @connectionId)

        --won't have much effect on this code, since the row will now be exclusively locked, and
        --the max will need to see the new row to be of any effect.
        exec @retval = sp_releaseapplock @Resource = 'applock$test', @LockMode = 'exclusive';

        --this releases the applock too
        commit transaction
    end try
    begin catch
        --if there is an error, rollback and display it.
        if @@trancount > 0
            rollback transaction
            select cast(error_number() as varchar(10)) + ':' + error_message()
    end catch

    Now, you can see up a few connections using this stored procedure, like this, varying the parameters to get more or less clashing. of values.  Running it in such a tight loop it is not surprising that two connections will often get the same value and then try to insert the value:

    waitfor time '23:46' --set for a time to run so multiple batches
    --can simultaneously execute
    go
    exec applock$test @@spid,<1=use applock, 0 = don't use applock>,'delay in hours:minutes:seconds.parts of seconds'
    go 10000 --runs the batch 10000 times in SSMS

    You will probably be amazed at how many clashes you get if you have applocks turned off. Doing 10000 on 3 connections on a Pentium 4 2.1 GHz Laptop, I got over 1000 clashes pretty much constantly. With applocks turned on, all rows were inserted in very close to the same amount of time.

    To solidify the point that every connection has to follow the rules, turn off applocks on a connection or two and see the havoc it will wreak.  The critical section will now no longer be honored by the one connection, and it will clash quickly, especially if you use any delay (that is more interesting when you use locks. Without them, it will clash nearly every time. 

    Metadata about computed columns

    To page 257 of my Pro SQL Server 2005 Database Design and Optimization, I want to add the following example about getting the metadata about computed columns.

    To get information about your computed column, you can use a couple of catalog views. The first is sys.columns, and the second is sys.computed_columns. As an example, using the tables that were created in the original text:

    --This code was from the original book:
    create schema alt
    go
    CREATE TABLE alt.testCalc
    (
          value varchar(10),
          valueCalc AS UPPER(value),
          value2 varchar(10)
    )
    GO
    INSERT INTO alt.testCalc --this name was wrong in the original text
    VALUES ('test','test2')
    GO
    SELECT *
    FROM alt.testCalc
    go

    You can determine when a column is a computed column by looking at the sys.columns view:

    select cast(name as varchar(20)) as name, is_computed
    from sys.columns
    where object_id('alt.testCalc') = object_id

    This returns:

    name                 is_computed
    -------------------- ------------
    value                0
    valueCalc            1
    value2               0     

    If you want to see the specific metadata about the computed column, you can use the sys.computed_columns view, which will give you several pieces of information, including if the column is persisted, and the definition of the computed column:

    select cast(name as varchar(20)) as name, is_persisted, definition
    from sys.computed_columns
    where object_id('alt.testCalc') = object_id

    Which in our case is:

    name                 is_persisted    definition
    -------------------- --------------- --------------------------------
    valueCalc            0               (upper([value]))

    October 16

    Hidden message contest

    Most contests are of the sort where you win a book that you don't have, probably by some sort of chance.  This contest, on the other hand, is for people who have purchased the book (or have a copy from a library, or just "borrowed" it from a coworker's desk, etc.)  If you own the book, you probably noticed that I tried to lighten up the topic with the occasional joke, or humorous tidbit.  Obviously the topic is pretty dry in and of itself, and I am a bit of a cut up, as any of the ten people who have come to hear me speak know. 

    So when I was during the final layout phase of the book, I thought I would have a bit of fun for later to insert a hidden message in the book.  Over the next 12 weeks, I am going to present on my website page for the book (http://drsql.org/ProSQLServerDatabaseDesign.aspx,) a series of twelve clues to find the hidden message, which is very inconspicuously hidden in the book for you to find, as a reward to those of you who really, really pay attention!  Or at least those who can follow a set of clues.

    Grand prize (1) - 250 GB WD USB Hard disk
    First Prize (1) - Bundle of Apress books  (Pro SQL Server 2005, Pro SQL Server 2005 Assemblies, Pro SQL Server 2005 Reporting Services, SQL Server 2005 T-SQL Recipes, as well as another copy of Pro SQL Server 2005 Database Design and Optimization to give to a friend, assuming you used your own copy to win the contest with!)
    Second prizes (10) - Apress T-Shirt

    I will review the answers and the first person to correctly identify the quote (including punctuation,) where the quote is from, the page number from the book, and what made this a hidden message, will get the grand prize.  The second will get the first prize, and so on.  If no one correctly identifies the information identified above, I will choose the closest response and award the prizes.  If persons get the correct answer, I will reveal that a winner has been found, but the names of the actual winners and hidden message will be revealed on December 27.

    Fine print: All decisions by the judges will be final.  First and Grand prizes will not be awarded to any person who has received a promotional copy of the book, nor persons employed by Compass Technology in Chesapeake VA.  Any winners from these two groups will be awarded Second Prize.  All submissions become the property of drsql.org and may be posted anonymously here or on the blog at drsql.spaces.live.com at any time. 

    August 12

    Cardinality Enforcement using a User-Defined Function

    I was answering questions the other day on the forums, when the question came up of enforcing cardinality of a relationship.  My answer was technically correct from a coding standpoint (I used an AFTER TRIGGER), but Umachandar Jayachandran (another of my heros from his classic webpage: http://www.umachandar.com/) added to what I said and taught me something (http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=607417&SiteID=1&mode=1).
     
    I was under the belief that CHECK constraints fired pre-data being placed into tables, but I was wrong.  Luckily in the book I can't find any place where I had actually been specific about the details of when constraints were executed, but it did lead me to add a couple of new examples, for the implementing of cardinality constraints. 
     
    The code and PDF of the entire addendum can be downloaded from http://drsql.org/ProSQLServerDatabaseDesign.aspx
     
    ---------------------------------------------------------------------------------
     
    On page 232 in Chapter 5, the following tip should be added to the end of the section
     
    Implementing the Design;Uniqueness Keys;Alternate Keys
     
    When you have a relationship that is of the required one-to-one cardinality type (for example, a table assigning one employee to a given office) you can use a UNIQUE CONSTRAINT on the column that must only have one value in it.  If the column is nullable, then see the Addendum section: Basic Table Creation; Uniqueness Keys; Selective Uniqueness for more information on how to implement this.) 
     
    Note: an example of this will be in the addendum in a section entitled: Constraints Based On Functions.
     
    On page 283, I want to add the following bullet:
     
    *  Cardinality enforcement - for example if you can only a certain number of a certain value in a column or set of columns

    On Page 286 in Chapter 6, add the following section (that will refer back to the newly added bullet):
     
    Constraints; Constraints Based on Functions; Cardinality Enforcement

    Sometimes, a business rule states that for some entity, you may have no more than N related entities of some sort.  So when we get to the task of implementing this in the database, there are a few possibilities.  I mentioned earlier in the addendum that you can enforce a 1-1 relationship using a unique constraint. 
     
    An example of this might be of employee to office assignments. In our scenario, the client makes the claim that an employee can only be assigned to one and only one office.  So I will build tables for each of these needs:
     
    CREATE TABLE employee
    (
     employeeId int NOT NULL CONSTRAINT PKemployee PRIMARY KEY,
     employeeNumber char(4) NOT NULL CONSTRAINT AKemployee_employeeNumber UNIQUE
    )
    go
    CREATE TABLE office
    (
     officeId int NOT NULL CONSTRAINT PKoffice PRIMARY KEY,
     officeNumber char(4) NOT NULL CONSTRAINT AKoffice_officeNumber UNIQUE,
    )
     
    And then one for the assignment.  You might think that I could have just placed a relationship from employee to office, and I could have.  But, then I needed to make this nullable, which will be less efficient, and may well not be flexible enough in the future (consider that foreshadowing)
     
    CREATE TABLE employeeOfficeAssignment
    (
           employeeId int,
           officeId  int,
           CONSTRAINT PKemployeeOfficeAssignment
                    PRIMARY KEY (employeeId, officeId),
           CONSTRAINT FKemployeeOfficeAssignment$assignsAnOfficeTo$employee
                    FOREIGN KEY (employeeId) REFERENCES employee(employeeId),
           CONSTRAINT FKemployeeOfficeAssignment$assignsAnOfficeTo$officeId
                    FOREIGN KEY (officeId) REFERENCES office(officeId)
    )
    Then we apply a UNIQUE constraint to the employeeId column.  Note that this does not keep multiple people from being assigned to the same office.  If that is a requirement, you can add another UNIQUE CONSTRAINT to the employee.
     
    ALTER TABLE employeeOfficeAssignment
        ADD CONSTRAINT AKemployeeOfficeAssignment_employee UNIQUE (employeeId)
     
    Then set up a couple of employees and several offices:
     
    INSERT employee(employeeId, employeeNumber)
    VALUES (1,'A001')
    INSERT employee(employeeId, employeeNumber)
    VALUES (2,'A002')
    INSERT employee(employeeId, employeeNumber)
    VALUES (3,'A003')

    INSERT INTO office(officeId,officeNumber)
    SELECT 1,'3001'
    UNION ALL
    SELECT 2,'3002'
    UNION ALL
    SELECT 3,'3003'
    Then assign offices A001 into 3001:
     
    insert into employeeOfficeAssignment(employeeId, officeId)
    values (1,1)
     
    Then A002 and A003 into 3003
     
    insert into employeeOfficeAssignment(employeeId, officeId)
    values (2,3)
    insert into employeeOfficeAssignment(employeeId, officeId)
    values (3,3)
     
    But putting A003 into 3002 as well raises the following error:
     
    insert into employeeOfficeAssignment(employeeId, officeId)
    values (3,2)
     
    Msg 2627, Level 14, State 1, Line 1
    Violation of UNIQUE KEY constraint 'AKemployeeOfficeAssignment_employee'. Cannot insert duplicate key in object 'dbo.employeeOfficeAssignment'.
    The statement has been terminated.
     
    Now, (seeing how this is a section on CHECK constraints) the client says, well, actually we have changed our minds, and now each employee can be in one or two offices, but no more than two.  This change is actually quite simple.  Drop the UNIQUE constraint:
     
    ALTER TABLE employeeOfficeAssignment
        DROP CONSTRAINT AKemployeeOfficeAssignment_employee
     
    This allows > 1 office assignments now.  Then we will create a user defined function that will return the number of times an employeeId shows up in the employeeOfficeAssignment table:
     
    CREATE FUNCTION employeeOfficeAssignment$cardinalityTwo
    ( @employeeId int)
    RETURNS BIT AS
     BEGIN
        RETURN (SELECT TOP 1 1
                FROM   employeeOfficeAssignment
                WHERE  employeeId = @employeeId
                HAVING count(*) > 2)
       END
     
    And add the check constraint, checking for a NULL return, which we would get if there was not any matching rows.  This works because the CHECK constraint is executed after the data is in the table (though before any triggers):
     
    ALTER TABLE employeeOfficeAssignment
        ADD CONSTRAINT CHKemployeeOfficeAssignment_employeeId_cardinalityTwo
              CHECK (dbo.employeeOfficeAssignment$cardinalityTwo(employeeId) is null)
     
    Now assigning A003 into 3002 is allowed:
     
    insert into employeeOfficeAssignment(employeeId, officeId)
    values (3,2)
     
    But putting A003 also into 3001 will not work, since the following data is in the table:
     
    SELECT *
    FROM   employeeOfficeAssignment
     
    employeeId  officeId
    ----------- -----------
    1           1
    2           3
    3           2
    3           3
     
    But adding the third office assignment to A003:
     
    insert into employeeOfficeAssignment(employeeId, officeId)
    values (3,1)
     
    Causes the following error:
     
    Msg 547, Level 16, State 0, Line 1
    The INSERT statement conflicted with the CHECK constraint "CHKemployeeOfficeAssignment_employeeId_cardinalityTwo". The conflict occurred in database "tempdb", table "dbo.employeeOfficeAssignment", column 'employeeId'.
    The statement has been terminated.
     
    You could easily extend this example to be parameter driven (in case the user changes their mind) or even data driven from a table so when they change their mind they can fix it. 
     
    August 03

    Reviews!

    Back when the year was but a wee little year, I wrote my New Year's resolutions, and one of them was: "I will respond to most reviews that I see on any site, and will link to them, even (perhaps, especially) bad ones."
     
    Well, I don't have much defending to to so far, but I do want to mention the ones that I have received.
     
    Just yesterday, a review was posted on one of my favorite sites, www.sql-server-performance.com.  The review is by a person whom I really respect, and it is quite good.  You can find it here: http://www.sql-server-performance.com/book_reviews/database_design_optimization.asp
     
    Next, there are three reviews as of today on Amazon, all of them good.  I know two of the reviewers, both of which have books coming out later this year or next.  The other person I do not know, which I appreciate as well. 
     
    Reviews are still my least favorite part of the writing process, but at least I have a little free time now :)
    July 15

    Examples from books versus real world

    I was setting up something on my database server the other day, using this book by a certain author who will remain unnamed (for now) and I just couldn't get one of the examples to work.  I could get the basic example to work fine enough, but there was just something missing when I got to my "real world" situation with an already set up database.
     
    The problem was, in this example, the writer used certificates to sign an object which allowed cross database security using a certificate based user.  This writer didn't mention that it only worked for executable modules, a subset of objects which did not include views.  Grrr.  To be fair, the example will work just fine, but not as I wanted it to.
     
    So I then follow another path to use cross database chaining.  This example fails me too.  Again, as written the sample works just fine, but....in my real world situation there were two things missing:
     
    1.  Guest account - Something I noted from this artcle from MVP Dan Guzman - http://www.sqlmag.com/Articles/ArticleID/25145/pg/2/2.html, and again something I should have noted.  On the test machine in the model db the guest account was enabled, so when the two dbs are created, the rest works.
     
    2. Because the same user created the databases, the owner had the same sid.  On my "real world" situation, the database was from a restore and had a different sid for the sa user.  Even looking at the query that showed the db owner didn't help because of this.
     
    The thing that really burns me up is that the author of this book is me :)  It is a clear representation of what happens when doing experiments in a lab environment or in the real world.  In the real world stuff is messy, in the lab, messiness is controlled.  A book is a bit like a lab experiment in that my goal is to show you the right way to do things, and some times I overlook the wrong way, and in the case of #2, the wrong way was just a symptom of something very common. 
     
    All of this to say that I will be adding some to the addendum, but my apologies for this example being not up to snuff.  I beg you to send me an email to drsql@hotmail.com any time you find bits of the book that don't make sense to you, or that don't quite work.  I will help you out any way that I can, well, to the extent that I can anyhow :)
    June 13

    My book is going to have a second printing

    This might mean that they ordered a very small amount of the book to start with, but I don't care, it is still encouraging news.  But I am not here to beam with pride (well, perhaps a little, but that isn't the only reason, I suppose,) I am here to ask a favor.  If you got the first printing and have any comments that might help me correct something and make it better for the next set of people who buy the book, please go here and enter some your errata, or just send me email at drsql@hotmail.com.
     
    Your help is appreciated for future readers, and part of why I blog here.  I will give you more information, articles, notes about other books, etc, and you tell my why my work stinks for you.  You can tell me if you love it, but be sure to tell others this too :) 
    June 05

    Index key ordering

    Ah, the things that BOL can teach you when you read it. Something that I left out of my book when defining indexes was a good excuse for index key ordering.  It was something that I hadn't really considered enough.  SQL Server can scan an index in either direction, so for the most part the order of the keys is not a big deal, especially when you have a single key. 
     
    To page 409, I need to make the following tweak.  The paragraph that says:
     
    "The <columnList> is a comma-delimited list of columns in the table. Each column can be
    specified either in ascending (ASC) or descending (DESC) order for each column, with ascending
    being the default (unless you’re strictly using the index for descending searches, ascending
    is generally fine). SQL Server can traverse the index in either direction for searches."
     
    Should have added to it:
     
    However, there can be some benefit to ordering the index keys when you have a composite index.  This will be covered later in the chapter when we discuss composite indexes.
     
    To page 423 (where the composite indexes section ends), I need to add a section called The Effect of Key Order
     
    While SQL Server can traverse an index in either direction (since it is a doubly linked list,) sometimes it can be valueable to sort the keys of an index to match the sort order of some desired output.  For example, consider the case where you want to look the hire dates of your employees, ordered by the date that they were hired, in descending order.
     
    So you execute the following query (in the AdventureWorks database):
     
    SELECT maritalStatus, hiredate
    FROM   HumanResources.Employee
    ORDER BY maritalStatus ASC, hireDate DESC

    The plan for this query is:
     
      |--Sort(ORDER BY:([AdventureWorks].[HumanResources].[Employee].[MaritalStatus] ASC,
                      [AdventureWorks].[HumanResources].[Employee].[HireDate] DESC))
           |--Clustered Index Scan(OBJECT:([AdventureWorks].[HumanResources].[Employee].
                     [PK_Employee_EmployeeID]))
     
    Next, create a typical index with the default (ascending) sort order:
     
    CREATE INDEX employee_maritalStatus_hireDate ON HumanResources.Employee(maritalStatus,hiredate)
    The plan changes to an Index Scan (since it can use the index to cover the query,) and still a sort.
     
        |--Sort(ORDER BY:([AdventureWorks].[HumanResources].[Employee].[MaritalStatus] ASC,
                        [AdventureWorks].[HumanResources].[Employee].[HireDate] DESC))
           |--Index Scan(OBJECT:([AdventureWorks].[HumanResources].[Employee].
                        [employee_maritalStatus_hireDate]))
     
    Better, but not perfect.  Change that to an index sorted in the direction that the output is desired in:
     
    DROP INDEX employee_maritalStatus_hireDate ON HumanResources.Employee
    GO
    CREATE INDEX employee_maritalStatus_hireDate ON HumanResources.Employee(maritalStatus ASC,hiredate DESC)

     
    Now the Sort is gone: 
     
    |--Index Scan(OBJECT:([AdventureWorks].[HumanResources].[Employee].
                 [employee_maritalStatus_hireDate]), ORDERED FORWARD)
     
    Of course, to be sure this has limited applicability, and is not necessarily the best thing to do even if it improves a query or two, since this is now an index that will need to be maintained, which may end up being more costly than paying for the clustered index scan.  It is however another tool in the belt to enhance query performance if this operation is done frequently.
     

     
     
    May 25

    The winner is....

    If you don't know what I am talking about, on my new website (www.drsql.org) I was running a contest for a free book if you sent feed back.  The key word in that sentence is "was" as John M. from  Charlotte, NC.  willl be getting a free copy courtesy of my personal stash. 
     
    I have a few other ideas for contests up my sleeve, including one that will reward you with something other than a copy of my book (since you will need a copy of the book to play :)  In the future, I will mention the contest here on the blog as well as the website for anyone interested. 
     
     
    May 24

    Duplicate Lists/Duplicate Data

    A good example of why to only keep one (normalized) list of information was apparent the past few days when I realized that I had overlapped two of my lists to send books to reviewers.  I had three lists:
     
    1.  People who had asked for one/helped me out in some way before it was finished (I like giving out stuff to people who have helped me out!)
    2.  Influential people that my marketing people would send to..
    3.  Copies that I was sending out from my stash (and my own nickel for shipping :)
     
    Despite my best work, I ended up duplicating two book sends, and even better both came from the publisher  and not from me.  One of the people who got double was on a different continent, so thank goodness I didn't send two from my stash.  Shipping costs to different continents gets expensive.  You can send most books via priority USPS post for less than 5 bucks including delivery costs.
     
    As the multi-book reciever noted, the sweet irony of having a book on DB design duplicated was more than I could pass up blogging about because it is true.  I had three lists to do pretty much the same thing.  I actually did design a database for this sort of thing AFTER I had sent the first two sets of books out, so I hope to never do this again (I am also going to publish that model as a case study sometime, so if you really want it, poke me with a virtual stick sometime). 
     
    The point of this is many organizations have just this sort of organization to their systems of replying/responding to people.  Spreadsheets, access databases, text files (I had a Word Document, and Outlook note, and a SQL Server 2005 Express Database (sadly, it came last)).  My list was only about 30 names so far (I am giving away one copy to anyone who follows the directions here right now.)  But it will grow.  I plan to have little contests all throughout the life of the book because it is fun and I like to give things away (of course I really like it when you exchange those little pieces of paper for the 672 pieces of paper, but that goes without saying, huh?)  
     
    Imagine how some companies with hundreds of contacts on 50 different Excel spreadsheets must feel.  Every time I have been involved in a system conversion spreadsheets always appear from nowhere as major data storage devices.  Using a relational database system is a great way to apply technology, make lives easier and happier (including your spouse when you buy her a fancy new vehicle from the big check you get for saving some company even more money, perhaps?)
     
    Part of the reason why I wanted to post this is because of how the person let me know. His email went something like:
     
    > Hi Louis,
    > Hi Louis,
    > And another big thank you for the bound copy.
    > And another big thank you for the bound copy.
    > In case you are wondering why I'm duplicating all my lines,
    > In case you are wondering why I'm duplicating all my lines,
    > it is because I received a copy of the book yesterday.
    > it is because I received another copy of the book today.
    Classic.  Either way another user group will get another book as a prize, so I don't mind.

    Customizing Index Locking

    Something that I hadn't seen yet was the new locking granularity settings on indexes.  It is a really interesting thing that now we have settings to revert back to "old-style" page locks on index access.   The syntax is simply part of the CREATE/ALTER INDEX command:
     
    ALTER INDEX <indexName>
      ON <tableName>
      SET (ALLOW_ROW_LOCKS = <OFF | ON>, ALLOW_PAGE_LOCKS = <OFF | ON>);
     
    I am going to add a section on this setting soon to the book addendum, but I want to make good and sure I understand the use cases for this setting.  I haven't seen any other people writing about it, and I only see a few basic uses for it (other than when doing low concurrency activites, or really high throughput stuff were page locking was always better.  But make sure it is low concurrency if you are doing any writes, because it could get ugly.)  If you have any ideas on how to use this feature, email me at drsql@hotmail.com and let me know.  I have considered that this might be a good idea for data warehouse loads, set the indexes to only do full object locks.  Since that operation is single threaded essentially (SQL Server can use more than one thread, but they won't block one another using SQL Server locks, only hardware latches and such).
     
    I chose the HumanResources.Employee table for the example, because it was small enough not to go into table lock mode for the simple tables scan I will use for the demonstration.  SQL Server as always can escalate the indexing chosen and turning off page locks will force the query processor to escalate directly to full object locks, which will single thread any write operations. 
     
    I will use the primary key of the table, so the clustered index scan will be the affected index operation. Just in case i am not the first person trying this, check the setting of allow_row_locks AND allow_page_locks settings before changing:
     
    SELECT allow_row_locks, allow_page_locks
    FROM sys.indexes
    WHERE name = N'PK_Employee_EmployeeId';
     
    The result should both be 1 or the following example will fail.  Read on for how to set it. To hold the locks that are aquired, you need to be in either REPEATABLE READ or SERIALIZABLE isolation level. (In READ COMMITTED locks are released after a rows IS fetched so it IS hard TO VIEW the results OF locking operations, a good thing IN a live system, a real pain FOR demonstrations!)
     
    SET TRANSACTION ISOLATION LEVEL REPEATABLE READ
    BEGIN TRANSACTION
    SELECT * FROM HumanResources.Employee
     
    In another connection, execute this query from get the locks being held (originated from a previous post):
     
    select  case des.transaction_isolation_level
                when 0 then 'Unspecified' when 1 then 'ReadUncomitted'
                when 2 then 'ReadCommitted' when 3 then 'Repeatable'
                when 4 then 'Serializable' when 5 then 'Snapshot'
            end as transaction_isolation_level,
            request_session_id, resource_type, request_mode,
            request_type
    from sys.dm_tran_locks dtl
            left outer join sys.dm_exec_sessions des
                on dtl.request_session_id = des.session_id
    where request_session_id <> @@spid
     
    Part OF what you will see IS:
     
    A shared database lock (preventing others from dropping it on you
     
    transaction_isolation_level request_session_id resource_type   request_mode   request_type
    --------------------------- ------------------ --------------- -------------- ------------                                         
    Repeatable                  51                 DATABASE        S              LOCK
     
    Intent shared locks on every page the query accesses, telling other connections that you might take a lock
     
    Repeatable                  51                 PAGE            IS             LOCK
     
    And a bunch of KEY locks, one for each row.
     
    Repeatable                  51                 KEY             S              LOCK
    Repeatable                  51                 KEY             S              LOCK
     
    Next, release these locks:
     
    ROLLBACK
     
    Now I will turn off row and page locks on the clustered index (the table)
     
    ALTER INDEX PK_Employee_EmployeeId
       ON HumanResources.Employee
       SET (ALLOW_ROW_LOCKS = OFF, ALLOW_PAGE_LOCKS = OFF);
     
    Checking the effects of this query:
     
    SELECT allow_row_locks, allow_page_locks
    FROM sys.indexes
    WHERE name = N'PK_Employee_EmployeeId';
     
    Returns:
     
    allow_row_locks allow_page_locks
    --------------- ----------------
    0               0
     
    Now EXECUTE the test query again:
     
    SET TRANSACTION ISOLATION LEVEL REPEATABLE READ
    BEGIN TRANSACTION
    SELECT * FROM HumanResources.Employee
     
    Execute the locking query again, AND you will get a far different locking pattern:
     
    The same database lock as before:

    transaction_isolation_level request_session_id resource_type   request_mode   request_type
    --------------------------- ------------------ --------------- -------------- -------------                                         
    Repeatable                  51                 DATABASE        S              LOCK

    But now, only an object lock:
    Repeatable                  51                 OBJECT          S              LOCK
     
    Finally, set it back TO allow these lock types (lest someone notices :)
     
    ALTER INDEX PK_SalesOrderHeader_SalesOrderID
        ON Sales.SalesOrderHeader
        SET (ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON);
    This is a very interesting tweak you can now make to and index on a table, even the clustered index. I will be doing more on this when I see a really good use, particularly one that is based on an OLTP need rather than an ETL/Reporting type usage.
    May 15

    Getting the current isolation level

    I am putting together a new website that will have an index of the blog (I fumbled around one too many times looking for the post: Seeing the currently executing statement) when I saw this post about getting the isolation level in 2000
     
    I had already planned to add a post for 2005, and this could be useful information if you are working with the sample code in the book (since I know I occasionally lose track of exactly where I am when i am working on an example sometimes) so here it is.  In 2005, there is a column in the dynamic managment views sys.dm_exec_sessions and sys.dm_exec_requests that will let you get the current isolation level

    select case transaction_isolation_level
                when 1 then 'Read Uncomitted'
                when 2 then 'Read Committed'
                when 3 then 'Repeatable Read'
                when 4 then 'Serializable'
                when 5 then 'Snapshot'
                else 'Unspecified'
            end
    from   sys.dm_exec_sessions --or sys.dm_exec_requests, depending on what you want
    where  session_id = @@spid
     
    This update is again located on the main page of the blog (and will be moving to the website when it is unveiled.(it is already active on the web but it needs a few finishing touches :) 
     

    DROP INDEX statement syntax change

    I was scripting a database on Friday to move from a 2005 server to a 2000 server, (which requires the occasional touching up anyhow for stuff that isn't in 2000 there is an option to set to only script 2000 features, but if I used it I wouldn't have even noticed. ) A future required change in syntax for dropping an index is coming. 

    In the book, on Page 411, I use the following syntax to drop an index:

     

    DROP INDEX produce.vegetable.Xvegetable_consistency

     

    This syntax still works, but the new syntax is:


    DROP INDEX <indexName> ON <objectName>

     

    In this case it would be:

     

    DROP INDEX Xvegetable_consistency ON produce.vegetable

     

    You can get the latest copy of the addendum here.

    May 12

    Changing the schema of a table

    When writing, it is imperative that you think not only in terms of what you do, but in what anyone might do.  In reality, this is really difficult.  The other day a coworker of mine asked: "How you do you move a table from one schema to another?" 
     
    "Duh, I don't know" I thought to myself, not wanting to admit that what seemed really obvious at this point (he wanted to move it from a users's schema to a dbo schema) had never even crossed my mind (my initial thought might have been "why didn't you design it for the correct schema?" but that is unrealistic.)
     
    Turns out the syntax is quite simple:
     
    ALTER SCHEMA <schemaName> TRANSFER <otherSchemaName>.<objectName>
     
    I will add to the next book update a section on this topic, because I need to consider a few more of the angles before I formalize this (like errors when you have the same object already in the schema (like constraints, triggers, and even security, etc)) but I wanted to just first think out loud about the reason for the omiting, and to show you the basic syntax.  This is the difference between blogging and proper publishing. Here, one simple concept to make you go "hmmm."  When I go to publish it, you have to go "hmmm, wow, really, and how about...yeah, okay..."
     
    As a quick example, say you have two schemas, fred and barney:
     
    use tempdb
    go
    create schema fred
    go
    create schema barney
    go
     
    Fred gets a swimming pool:
     
    create table fred.swimmingPool
    (
        swimmingPoolId int primary key,
        poolStyle   varchar(10),
        etc         varchar(10)
    )
     
    Then it turns out that it should be Barney's:
     
    ALTER SCHEMA barney TRANSFER fred.swimmingPool
     
    Checking out the data again, you will see that the schema has changed:
     
    select *
    from fred.swimmingPool
    go   
    select *
    from barney.swimmingPool

    This returns:

     

    Msg 208, Level 16, State 1, Line 1
    Invalid object name 'fred.swimmingPool'.
     
    swimmingPoolId poolStyle  etc
    -------------- ---------- ----------
     
    Of course Fred would have never stood for this until the Betty and Wilma schemas got involved and they each shared half of the table.
     
    Louis would like to apologize for the previous example.  It is truthfully quite silly.  If you have no idea who Fred, Barney, Wilma or Betty are, they are characters from the series, The Flintstones. 
    May 07

    Update: Selective uniqueness

    To page 228, I need to add a section on how to deal with the situation where you need to enforce uniqueness on a column (or set of columns) where not every value may be unique (like if you want to allow more than 1 NULL value, or some other value.
     
    I had thought of this many times, but just completely forgot to add it.  My brain was poked with a stick when I saw this really wonderful post by the Microsoft SQL Server Development Customer Advisory Team entitled Achieving Selective Uniqueness in SQL Server Tables.
     
    Edit: Oops, left (in progress) in the title.  Some of these articles were started weeks ago :)
     
    --------------------------------------- 

    We previously discussed PRIMARY KEY and UNIQUE constraints, but in some situations neither of these will exactly fit the situation.  For example, you may need to make sure some subset of the data is unique, rather than every row.  In a SQL Server unique index, you may only have a single null row, which can be troublesome.  An example of this is a one-to-one relationship where you need to allow nulls, for example a customerSettings table that lets you add a row for optional settings for a customer. If a user has settings, a row is created, but you want to insure that only one row is created. 

    There are a couple of ways to do this (such as in a trigger or stored procedure using an EXISTS query) but the easiest way is to use an indexed view.  (Note: all versions of SQL Server can use indexed views, but the Enterprise Edition makes special use of them.  For more information see chapter 8, Indexing.)

    For example, say you have an employee table and each employee can possibly have an insurance policy.  The policy numbers must be unique, but the user might not have a policy.  First, create the table:

    CREATE TABLE dbo.employee
    (
        employeeId int identity(1,1) constraint PKdbo_employee primary key,
        employeeNumber char(5) not null constraint AKdbo_employee_employeeNubmer UNIQUE,
        --skipping other columns
        insurancePolicyNumber char(10) null
    )

    Then create a view that returns all rows other than null insurancePolicyNumber values.  Note that it has to be schema bound to allow for indexing:

    CREATE VIEW dbo.employee_insurancePolicyNumberUniqueness
    WITH
    SCHEMABINDING
    AS
        SELECT  insurancePolicyNumber
        FROM    dbo.employee
        WHERE  insurancePolicyNumber is not null

    Then create a unique, clustered index on the view:

    CREATE UNIQUE CLUSTERED INDEX insurancePolicyNumber
        ON employee_insurancePolicyNumberUniqueness(insurancePolicyNumber)

    Then create an initial sample row:

    INSERT INTO employee (employeeNumber, insurancePolicyNumber)
    VALUES ('A0001','1111111111')

    Upon attempting to give another employee the same insurancePolicyNumber:

    INSERT INTO employee (employeeNumber, insurancePolicyNumber)
    VALUES ('A0002','1111111111')

    This fails:

    Msg 2601, Level 14, State 1, Line 1
    Cannot insert duplicate key row in object 'dbo.employee_insurancePolicyNumberUniqueness' with unique index 'insurancePolicyNumber'.
    The statement has been terminated.

    However adding two rows with null will work fine:

    INSERT INTO employee (employeeNumber, insurancePolicyNumber)
    SELECT 'A0006','2222222222'
    UNION ALL
    SELECT 'A0007',NULL
    UNION ALL
    SELECT 'A0008',NULL

    You can see this:

    SELECT *
    FROM   dbo.employee

    This returns:

    employeeId  employeeNumber insurancePolicyNumber
    ----------- -------------- ---------------------
    1           A0001          1111111111
    3           A0003          2222222222
    4           A0004          NULL
    5           A0005          NULL

    As another example, consider the case where you want to ensure that only a single row is set as primary for a group of rows, such as a primary contact for an account.
     
    CREATE TABLE dbo.accountContact
    (
        contactId   varchar(10) not null,
        accountNumber   char(5) not null, --would be FK
        primaryContactFlag bit not null,
        constraint PKdbo_accountContact
            primary key(contactId, accountNumber)
    )

     

    Again, create an index, only this time only choose rows with primaryContactFlag = 1.  The other values in the table could have as many other values as you want (of course in this case since it is a bit the values could only be 0 or 1):

    CREATE VIEW accountContact_primaryContactUniqueness
    WITH
    SCHEMABINDING
    AS
        SELECT  accountNumber
        FROM    dbo.accountContact
        WHERE   primaryContactFlag = 1

    And the clustered unique index:
       
    CREATE UNIQUE CLUSTERED INDEX contactId
        ON accountContact_primaryContactUniqueness(accountNumber)

    So if you try to insert two rows that are primary, in the following for account number '11111':

    INSERT INTO dbo.accountContact
    SELECT 'bob','11111',1
    go
    INSERT INTO dbo.accountContact
    SELECT 'fred','11111',1

    The following error is returned:

    Msg 2601, Level 14, State 1, Line 1
    Cannot insert duplicate key row in object 'dbo.accountContact_primaryContactUniqueness' with unique index 'contactId'.
    The statement has been terminated.

    So to insert this data you will need to update the the other row to be not primary, and then insert the new primary row:

    BEGIN TRANSACTION

    UPDATE dbo.accountContact
    SET primaryContactFlag = 0
    WHERE  accountNumber = '11111'

    INSERT dbo.accountContact
    SELECT 'fred','11111',1

    COMMIT TRANSACTION

    Note that in cases like this you would definitely want to use a transaction so you don't end up without a primary contact if the insert fails for some reason.

     

    -------------------------------------------

     
    This change (and any others I add) can be downloaded here, as well as linked in the downloads section of the blog: Pro SQL Server 2005 Database Design and Optimization Addendum

     

    May 04

    Full Sample Chapter Available

    I don't want to keep doing DesignBook posts, as they are probably boring some of you, but I just want to note that the sample chapter on the Apress site is not a full chapter instead of just part of a chapter. 
     
    May 02

    A sample chapter is available

    Plus, the book is one of the featured titles on the Apress site (http://www.apress.com/).
     
    The sample chapter excerpt is Chapter 2, Data Modeling where I cover the IDEF1X modeling language used by most modeling tools (it is the one with the black dots on the child end of relationship lines, instead of "crow's feet" which Information Engineering uses).  When I get around to publishing some extra sample materials I am going to use the Information Engineering methodology style diagrams to give me some more experience, as well as make it more accessible to those of you who like that style.  The end of the chapter which is not part of the excerpt does cover the high points of the crow's feet style of model.
     
    The sample chapter excerpt can be downloaded here, along with the source code, table of contents, etc.   I also have provided the table of contents here: Table Of Contents as well as my addendum that I will be working on to add to the materials over time.
    May 01

    Update: Instead of Trigger Enhancement

     
    A change that I didn't notice they had made to Instead of Triggers is going to be my first book update.  It is one of those tiny changes that didn't really get mentioned, and I only heard of it because the illustriuous Kalen Delaney noticed the change when researching a newsgroup poster's issue. 
     

    Adds to the information starting on page 312

     

    NOT NULL constraints are not checked before an instead of trigger, as they were in 2000.  This comes into play when you have columns declared as NOT NULL and you have an instead of trigger.  If the trigger's purpose is to set the value, in 2000 you had to have a default the value to some useless value, either with a default constraint or by setting a value in the INSERT statement.  A NOT NULL column that has the identity value did not need to be defaulted in 2000 SP3, which I have to test with.

      

    As an example, consider the following table:

     

    CREATE TABLE insteadOfTriggerTest
    (
        insteadOfTriggerTestId int not null identity(1,1)
                constraint PKinsteadOfTriggerTest primary key,
        name    varchar(30) not null
                constraint AKname unique,
        rowCreateDate datetime not null
    )
     

    I then add the following instead of trigger to automatically set the rowCreateDate value (I would also have one for a rowModifyDate in a real system.)

     

    CREATE TRIGGER insteadOfTriggerTest$insteadOfInsert
    ON insteadOfTriggerTest
    INSTEAD OF INSERT

    AS
     BEGIN
        SET NOCOUNT ON
        INSERT insteadOfTriggerTest(name, rowCreateDate)
        SELECT name, getdate()
        FROM inserted
           
     END
     

    Try to execute this on a 2000 server:

     

    INSERT insteadOfTriggerTest(name)
    SELECT 'bob'

     

    This returns an error:

     

    Msg 233, Level 16, State 2, Line 1
    The column 'rowCreateDate' in table 'insteadOfTriggerTest' cannot be null.

     

    On 2005, it works fine:

     

    SELECT *
    FROM   insteadOfTriggerTest

     

    insteadOfTriggerTestId name                           rowCreateDate
    ---------------------- ------------------------------ -----------------------
    1                      bob                            2006-04-23 21:30:13.137


    This is a great change and makes using instead of triggers even easier as a mechanism to maintain automatically generated values.

     

    Note:

    Unfortunately, scope_identity() still returns NULL, since the actual insert was in the instead of trigger:

     

    INSERT insteadOfTriggerTest(name)

    SELECT 'fred'

     

    SELECT scope_identity()

     

    Returns:

     

    ---------------------------------------

    NULL

     

    Even using the new OUTPUT clause doesn’t work either:

     

    DECLARE @table table (value int)

    INSERT insteadOfTriggerTest(name)

    OUTPUT inserted.insteadOfTriggerTestId into @table

    SELECT 'barney'

     

    SELECT *

    FROM   @table

     

    This returns:

     

    value

    -----------

    0

     

    Of course since the name column is an alternate key, the identity value can be retrieved by searching on the value inserted.

     

    SELECT insteadOfTriggerTestId

    FROM   insteadOfTriggerTest

    WHERE  name = 'barney'

     
     
    This change (and any others I add) can be downloaded here, as well as linked in the downloads section of the blog:

     

     

    April 27

    My books arrived!

    So that means that the books are actually real now.  They look pretty good (which is to say they look just like all of the other Apress books! 
     
    Interestingly enough, the shipping date on Amazon now says "Usually ships within 2 to 5 weeks."  Just yesterday it was "12-14 days."  Barnes and Noble says: "usually ships within 2-3 days."  Bookpool claims it hasn't been published yet,  As I hear more, I will let you know.
     
    It is available immediately as a downloadable e-book on the Apress site, if you are into the whole e-book thing.  It is nice to be able to search for INSERT, or INSTEAD OF and find all references directly (and clickable.)  Not to mention you can copy code directly as well as use the code download.
     
     
    April 23

    Tomorrow's the Day

    I have been assured that the book should be released tomorrow, so if you have preordered the book, it should be on its way soon enough.  I have been out of town for the past few days, so I am not even sure if I have my own copy yet (they could be on my desk at work :)