Jean-Charles

The cdc capture job ran into errors on our prod server. The error reporting in dm_cdc_errors is

Cannot insert duplicate key row in object 'cdc.CI_dbo_valuation_CT' 
WITH unique index 'CI_dbo_valuation_t_CT_clustered_idx'. 
The duplicate key value is (0x005e624e00063c60002c, 1, 0x005e624e00063740005c, 2).

The index definition is on the following columns. The following selects shows that there is already a row that was captured. 

SELECT * FROM cdc.CI_dbo_valuation_CT
WHERE [__$start_lsn] = 0x005e624e00063c60002c
AND [__$command_id] = 1
AND[__$seqval] = 0x005e624e00063740005c
AND[__$operation] = 2

My google search showed that there is a bug with sql2016 when Merge statement used with cdc, capture job fails. This was rectified in CU1. https://support.microsoft.com/en-us/help/3155503/fix-merge-statement-to-sync-tables-is-unsuccessful-when-change-data-ca

But we are running sql2016 CU11 and we are seeing this error. Restarting the CDC capture job fixed the error and capture job was able to move past the LSNs. I am trying to find out the root cause of this problem.

Any help would be great. 

Hello Everyone,

  Hope everyone is staying safe & Healthy in SQL Community.

I am looking some advise if someone can suggest best of it. 

 Currently I have SQL Server Shared database Environment with over 250 DB. Current shared environment has 32 cores , 280 GB RAM.

I need to move 5 DB's to brand new  SQL Server 2017 and for that i need to come up with CORE + MEmory recommendation. 

Core count specially needed to save cost for the SQL License. 

what formula or process shall i use to recommend core count and Memory. 

Current CPU %

DB    CPU % 

DB1 76.54
DB2 16.36
DB3 16.29
DB4 10.80
DB5 10.00
DB6 8.00

Current Memory usage

let me know if you need more information. 

Thank you very much for your time and effort to answer this post. Please Mark As Answer if it is helpful. \\Aim To Inspire Rather to Teach Best -Ankit

Hi we are migrating to 2019 enterprise.  And believe MS's data masking feature will be of interest.

does anyone have experience comparing that feature with say Red Gate's or other 3rd party masking products? 

We do not think scrambling will be important.  and we do not think encryption will be important.  But i m running out those questions with our compliance people. 

Hi,

I am trying to delete file and file groups from database as they are not required. I was able to delete the file, however while deleting the filegroup I get the below message. I tried deleting the filegroup from SSMS and there also I get the below message.

Msg 5042, Level 16, State 12, Line 6 The filegroup '*****' cannot be removed because it is not empty.

I already tried below commands so that I can delete the filegroup

Swapnil Ambre

Hi, 

By default sql tries to restart on the same node in case of a failure, next it does failover.

Personally I am good with the default behavior , but I am being asked a interesting question - is it possible to have sql NOT try a restart on the same node and just failover in case of a failure?

And where is it done from, is it this:

In cluadmin-SQL Server Properties-Policies- 

There is a option

1. Max restarts in the specified period - Will making it zero do the needful?

Or there is some other way? Thanks.

D

Good Morning Experts,

how to add a data file to the primary database in Log Shipping without breaking log shipping

Kiran

if there are table having

how to change the setting of this table ?

Running this way takes 40 seconds

SET TRANSACTION ISOLATION LEVEL

READ UNCOMMITTED 
SET NOCOUNT ON


DECLARE @CurrentDate datetime, @minOrderId1 int, @minOrderId2 int

SET @CurrentDate = ags.dbo.uf_strip_time(GETDATE())
select @minOrderId1 = min(orderid) from epos_orders where xmitdate >= dateadd(dd,-2,@CurrentDate)  --back 2 days
select @minOrderId2 = min(orderid) from epos_orders where xmitdate >= dateadd(wk,-1,@CurrentDate)  --back 1 week

CREATE TABLE #dupes
            (   rowId       int identity,
                store       int,
                partition1  varchar(3),
                Conf1       int,
                XmitDate1   datetime,
                ItmCnt1     int,
                TotQty1     int,
                XferFileId1 int,
                partition2  varchar(3),
                Conf2       int,
                XmitDate2   datetime,
                ItmCnt2     int,
                TotQty2     int,
                XferFileId2 int
            )

----------------------------------------------------------------------------------------------------------
-- identify possible dupes within all files that were just imported (status='' or 'X')
INSERT #Dupes
   SELECT
               store        = t1.store,
               partition1   = t1.partition,
               Conf1        = t1.conf,
               XmitDate1    = t1.xmitdate,
               ItmCnt1      = t1.ItmCnt,
               TotQty1      = t1.TotQty,
               XferFileId1  = t1.xferfileId,
               partition2   = t2.partition,
               Conf2        = t2.conf,
               XmitDate2    = t2.xmitdate,
               ItmCnt2      = t2.ItmCnt,
               TotQty2      = t2.TotQty,
               XferFileId2  = t2.xferfileId

     FROM


           (    select  -- transmissions not sent to OMI yet
                          xferfileId, conf, eo.store, partition, xmitdate, ItmCnt= count(item), TotQty= sum(qty)
                    from  epos_orders eo
                   where  1=1
                          and orderID >= @minOrderId1
                          --status = '' means order just loaded; hasn't been thru edits yet
                          --status = 'X' means order passed edits; ready to be exported to OMI
                          and (status = '' or cast(status as varbinary(1)) = cast('X' as varbinary(1)) )
                          and storeOverride <> '99' --used specifically to prevent order from getting caught by this process
                          and eo.store not in (995, 1995, 2995, 3995) --cash n carry
                group by  xferfileId, conf, eo.store, partition, xmitdate
                  having  count(item) >= 5
            ) T1
INNER JOIN
           (    select  -- transmissions not sent to OMI yet
                          xferfileId, conf, eo.store, partition, xmitdate, ItmCnt= count(item), TotQty= sum(qty)
                    from  epos_orders eo
                   where  1=1
                          and orderID >= @minOrderId1
                          --status = '' means order just loaded; hasn't been thru edits yet
                          --status = 'X' means order passed edits; ready to be exported to OMI
                          and (status = '' or cast(status as varbinary(1)) = cast('X' as varbinary(1)) )
                          and storeOverride <> '99' --used specifically to prevent order from getting caught by this process
                          and eo.store not in (995, 1995, 2995, 3995) --cash n carry
                group by  xferfileId, conf, eo.store, partition, xmitdate
                having count(item) >= 5
            ) T2
        ON  T1.store = t2.store
            and T1.ItmCnt = T2.ItmCnt
            and T1.TotQty = T2.TotQty
     WHERE  --1st condition: duplicate is in same file, different partition
            (T1.xferFileId = T2.xferFileId
             and T1.partition <> T2.partition)
             OR
             --2nd condition: duplicate is different file, same partition
            (T1.xferFileId <> T2.xferFileId
             and T1.partition = T2.partition)
             OR
             --3rd condition: duplicate is different file, different partition
            (T1.xferFileId <> T2.xferFileId
             and T1.partition <> T2.partition)
   ORDER BY t1.xferFileId, t1.store, t1.conf, T1.partition

-- use for debugging
--select * from #dupes

delete #dupes
  from #dupes d1
 where exists (select *  from #dupes d2 
                where    --same file is in more than 1 row
                          (d1.XferFileId1 = d2.XferFileId1
                           and d1.partition2 = d2.partition1
                           and d2.rowId > d1.rowId)
                       OR --extra rows resulting from cartesian product
                          (d1.XferFileId1 = d2.XferFileId2
                           and d1.partition1 = d2.partition2
                           and d2.rowId > d1.rowId)
                      )
-- use for debugging
--select * from #dupes                      
----------------------------------------------------------------------------------------------------------


----------------------------------------------------------------------------------------------------------
--identify possible dupes against files that have already been processed
INSERT #Dupes
   SELECT
               store        = TODAY.store,
               partition1  =  TODAY.partition,
               Conf1        = TODAY.conf,
               XmitDate1    = TODAY.xmitdate,
               ItmCnt1      = TODAY.ItmCnt,
               TotQty1      = TODAY.TotQty,
               XferFileId1  = TODAY.xferFileId,
               partition2   = PASTWEEK.partition,
               Conf2        = PASTWEEK.conf,
               XmitDate2    = PASTWEEK.xmitdate,
               ItmCnt2      = PASTWEEK.ItmCnt,
               TotQty2      = PASTWEEK.TotQty,
               XferFileId2  = PASTWEEK.xferFileId

     FROM


              (   select  -- transmissions not sent to OMI yet
                          conf, eo.store, partition, xmitdate, xferFileId, ItmCnt= count(item), 
                          TotQty= sum(eo.qty + isnull(gbx.qty,0) + isnull(gbr.qty,0) )
                    from  epos_orders eo
               left join OrderManagement..GroupBookingXref gbx
                      on  gbx.origOrderId = eo.OrderID
               left join OrderManagement..GroupStoreBkgReorderReductions gbr
                      on gbr.origOrderId = eo.OrderID      
                   where  1=1
                          and orderID >= @minOrderId1
                          --status = '' means order just loaded; hasn't been thru edits yet
                          --status = 'X' means order passed edits; ready to be exported to OMI
                          and (status = '' or cast(status as varbinary(1)) = cast('X' as varbinary(1)) )
                          and storeOverride <> '99' --used specifically to prevent order from getting caught by this process
                          and eo.store not in (995, 1995, 2995, 3995) --cash n carry
                group by  conf, eo.store, partition, xmitdate, xferFileId
                  having  count(item) >= 5
             )  as TODAY
INNER JOIN
             (    select  -- transmissions from past week
                          conf, eo.store, partition, xmitdate, xferFileId, ItmCnt= count(item), 
                          TotQty= sum(eo.qty + isnull(gbx.qty,0) + isnull(gbr.qty,0) )
                    from  epos_orders eo
               left join OrderManagement..GroupBookingXref gbx
                      on  gbx.origOrderId = eo.OrderID
               left join OrderManagement..GroupStoreBkgReorderReductions gbr
                      on gbr.origOrderId = eo.OrderID      
                   where  1=1
                          and orderID between @minOrderId2 and @minOrderId1 
                          and cast(status as varbinary(1)) <> cast('e' as varbinary(1))
                          and eo.store not in (995, 1995, 2995, 3995) --cash n carry
                group by  conf, eo.store, partition, xmitdate, xferFileId
                  having  count(item) >= 5
             )  as PASTWEEK
        ON      TODAY.store       = PASTWEEK.store
            and TODAY.[partition] = PASTWEEK.[partition]
            and TODAY.ItmCnt      = PASTWEEK.ItmCnt
            and TODAY.TotQty      = PASTWEEK.TotQty

     WHERE  PASTWEEK.xmitdate < TODAY.xmitdate

-- use for debugging
--select * from #dupes

-- ...an order is a duplicate IF: store, partition, item count & item qty are same
--    as the lastly submitted partition by the same store (this is the tricky part)

-- file must be the most recently submitted order prior to the current one
DELETE #dupes
  FROM #dupes d1
 WHERE 1=1
   AND  xmitdate2  <> (select max(d2.xmitdate2)
                         from #dupes d2
                        where d2.store = d1.store
                          and d2.partition1 = d1.partition1)

-- use for debugging
--select * from #dupes

-- If store has submitted at least 1 file w/same partition since the older file - exclude
DELETE #dupes
    FROM #dupes d1
     WHERE 1=1
     and exists (select top 1 *
                   from Epos_Orders eo
                  where 1=1
                    and eo.OrderID between @minOrderId2 and @minOrderId1 
                    and eo.store = d1.store
                    and eo.partition = d1.partition1
                    and eo.XmitDate between d1.XmitDate2 and d1.XmitDate1
                    and cast(eo.conf as int) <> d1.Conf1
                    and cast(eo.conf as int) <> d1.conf2
                    and cast(eo.status as varbinary(1)) <> cast('e' as varbinary(1))
                 )


--if running interactively, return the dupes
--insert #dupes
--    select 401, 7, 1212, '8/3/2016 08:15:00', 15, 99, 842010, 1212, '08/3/2016 07:30:00', 15, 99, 842004

select * from #dupes

drop table #dupes
------------------------------------------------------------------------------------------------

GO

running this way takes 2 seconds

SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED 
SET NOCOUNT ON


CREATE TABLE #dupes
            (   rowId       int identity,
                store       int,
                partition1  varchar(3),
                Conf1       int,
                XmitDate1   datetime,
                ItmCnt1     int,
                TotQty1     int,
                XferFileId1 int,
                partition2  varchar(3),
                Conf2       int,
                XmitDate2   datetime,
                ItmCnt2     int,
                TotQty2     int,
                XferFileId2 int
            )

CREATE TABLE #today
            (   
                store       int,
                partition  varchar(3),
                Conf       int,
                XmitDate   datetime,
                ItmCnt     int,
                TotQty     int,
                XferFileId int
            )

CREATE TABLE #lastWeek
            (   
                store       int,
                partition  varchar(3),
                Conf       int,
                XmitDate   datetime,
                ItmCnt     int,
                TotQty     int,
                XferFileId int
            )


DECLARE @CurrentDate datetime, @minOrderId1 int, @minOrderId2 int

SET @CurrentDate = ags.dbo.uf_strip_time(GETDATE())
select @minOrderId1 = min(orderid) from epos_orders where xmitdate >= dateadd(dd,-2,@CurrentDate)  --back 2 days
select @minOrderId2 = min(orderid) from epos_orders where xmitdate >= dateadd(wk,-1,@CurrentDate)  --back 1 week

----------------------------------------------------------------------------------------------------------
-- identify possible dupes within all files that were just imported (status='' or 'X')
INSERT #Dupes
   SELECT
               store        = t1.store,
               partition1   = t1.partition,
               Conf1        = t1.conf,
               XmitDate1    = t1.xmitdate,
               ItmCnt1      = t1.ItmCnt,
               TotQty1      = t1.TotQty,
               XferFileId1  = t1.xferfileId,
               partition2   = t2.partition,
               Conf2        = t2.conf,
               XmitDate2    = t2.xmitdate,
               ItmCnt2      = t2.ItmCnt,
               TotQty2      = t2.TotQty,
               XferFileId2  = t2.xferfileId

     FROM


           (    select  -- transmissions not sent to OMI yet
                          xferfileId, conf, eo.store, partition, xmitdate, ItmCnt= count(item), TotQty= sum(qty)
                    from  epos_orders eo
                   where  1=1
                          and orderID >= @minOrderId1
                          --status = '' means order just loaded; hasn't been thru edits yet
                          --status = 'X' means order passed edits; ready to be exported to OMI
                          and (status = '' or cast(status as varbinary(1)) = cast('X' as varbinary(1)) )
                          and storeOverride <> '99' --used specifically to prevent order from getting caught by this process
                          and eo.store not in (995, 1995, 2995, 3995) --cash n carry
                group by  xferfileId, conf, eo.store, partition, xmitdate
                  having  count(item) >= 5
            ) T1
INNER JOIN
           (    select  -- transmissions not sent to OMI yet
                          xferfileId, conf, eo.store, partition, xmitdate, ItmCnt= count(item), TotQty= sum(qty)
                    from  epos_orders eo
                   where  1=1
                          and orderID >= @minOrderId1
                          --status = '' means order just loaded; hasn't been thru edits yet
                          --status = 'X' means order passed edits; ready to be exported to OMI
                          and (status = '' or cast(status as varbinary(1)) = cast('X' as varbinary(1)) )
                          and storeOverride <> '99' --used specifically to prevent order from getting caught by this process
                          and eo.store not in (995, 1995, 2995, 3995) --cash n carry
                group by  xferfileId, conf, eo.store, partition, xmitdate
                having count(item) >= 5
            ) T2
        ON  T1.store = t2.store
            and T1.ItmCnt = T2.ItmCnt
            and T1.TotQty = T2.TotQty
     WHERE  --1st condition: duplicate is in same file, different partition
            (T1.xferFileId = T2.xferFileId
             and T1.partition <> T2.partition)
             OR
             --2nd condition: duplicate is different file, same partition
            (T1.xferFileId <> T2.xferFileId
             and T1.partition = T2.partition)
             OR
             --3rd condition: duplicate is different file, different partition
            (T1.xferFileId <> T2.xferFileId
             and T1.partition <> T2.partition)
   ORDER BY t1.xferFileId, t1.store, t1.conf, T1.partition
    --select '1', * from #dupes

   delete #dupes
  from #dupes d1
 where exists (select *  from #dupes d2 
                where    --same file is in more than 1 row
                          (d1.XferFileId1 = d2.XferFileId1
                           and d1.partition2 = d2.partition1
                           and d2.rowId > d1.rowId)
                       OR --extra rows resulting from cartesian product
                          (d1.XferFileId1 = d2.XferFileId2
                           and d1.partition1 = d2.partition2
                           and d2.rowId > d1.rowId)
                      )


    --select '2', * from #dupes
-------------------------------------------------------


----------------------------------------------------------------------------------------------------------
        --identify possible dupes against files that have already been processed
        INSERT #today
            select  -- transmissions not sent to OMI yet
                    eo.store, partition, conf, xmitdate, ItmCnt= count(item), 
                    TotQty= sum(eo.qty + isnull(gbx.qty,0) + isnull(gbr.qty,0) ), xferFileId
             from  epos_orders eo
        left join OrderManagement..GroupBookingXref gbx
               on  gbx.origOrderId = eo.OrderID
        left join OrderManagement..GroupStoreBkgReorderReductions gbr
               on gbr.origOrderId = eo.OrderID      
            where  1=1
                    and orderID >= @minOrderId1
                    --status = '' means order just loaded; hasn't been thru edits yet
                    --status = 'X' means order passed edits; ready to be exported to OMI
                    and (status = '' or cast(status as varbinary(1)) = cast('X' as varbinary(1)) )
                    and storeOverride <> '99' --used specifically to prevent order from getting caught by this process
                    and eo.store not in (995, 1995, 2995, 3995) --cash n carry
        group by  conf, eo.store, partition, xmitdate, xferFileId
          having  count(item) >= 5
        --select * from #today order by store

        INSERT #lastWeek
                select  -- transmissions from past week
                        eo.store, partition, conf, xmitdate, ItmCnt= count(item), 
                        TotQty= sum(eo.qty + isnull(gbx.qty,0) + isnull(gbr.qty,0) ), xferFileId
                 from  epos_orders eo
            left join  OrderManagement..GroupBookingXref gbx
                   on  gbx.origOrderId = eo.OrderID
            left join  OrderManagement..GroupStoreBkgReorderReductions gbr
                   on  gbr.origOrderId = eo.OrderID      
                where  1=1
                       and orderID between @minOrderId2 and @minOrderId1 
                       and cast(status as varbinary(1)) <> cast('e' as varbinary(1))
                       and eo.store not in (995, 1995, 2995, 3995) --cash n carry
             group by  conf, eo.store, partition, xmitdate, xferFileId
               having  count(item) >= 5
        --select * from #lastWeek order by store

       INSERT #Dupes
               SELECT
                           store        = TODAY.store,
                           partition1  =  TODAY.partition,
                           Conf1        = TODAY.conf,
                           XmitDate1    = TODAY.xmitdate,
                           ItmCnt1      = TODAY.ItmCnt,
                           TotQty1      = TODAY.TotQty,
                           XferFileId1  = TODAY.xferFileId,
                           partition2   = PASTWEEK.partition,
                           Conf2        = PASTWEEK.conf,
                           XmitDate2    = PASTWEEK.xmitdate,
                           ItmCnt2      = PASTWEEK.ItmCnt,
                           TotQty2      = PASTWEEK.TotQty,
                           XferFileId2  = PASTWEEK.xferFileId

                 FROM #today    today
           INNER JOIN #lastWeek pastweek
                ON      TODAY.store       = PASTWEEK.store
                    and TODAY.[partition] = PASTWEEK.[partition]
                    and TODAY.ItmCnt      = PASTWEEK.ItmCnt
                    and TODAY.TotQty      = PASTWEEK.TotQty
                 WHERE  PASTWEEK.xmitdate < TODAY.xmitdate

        --select '3', * from #dupes

        -- file must be the most recently submitted order prior to the current one
        DELETE #dupes
          FROM #dupes d1
         WHERE 1=1
           AND  xmitdate2  <> (select max(d2.xmitdate2)
                                 from #dupes d2
                                where d2.store = d1.store
                                  and d2.partition1 = d1.partition1)

        -- use for debugging
        --select '4', * from #dupes

        -- If store has submitted at least 1 file w/same partition since the older file - exclude
        DELETE #dupes
            FROM #dupes d1
             WHERE 1=1
             and exists (select top 1 *
                           from Epos_Orders eo
                          where 1=1
                            and eo.OrderID between @minOrderId2 and @minOrderId1 
                            and eo.store = d1.store
                            and eo.partition = d1.partition1
                            and eo.XmitDate between d1.XmitDate2 and d1.XmitDate1
                            and cast(eo.conf as int) <> d1.Conf1
                            and cast(eo.conf as int) <> d1.conf2
                            and cast(eo.status as varbinary(1)) <> cast('e' as varbinary(1))
                         )
----------------------------------------------------------------------------------------------------------


select * from #dupes

DROP TABLE #today
DROP TABLE #lastweek
DROP TABLE #dupes
GO

We would prefer it to run the first way but it gets hung up in the Insert #Dupes. We have looked at indexes and all are fine. We have run update_statistics just to be sure we were getting the best execution plan. We have changed cost threshold for parallelism from 5 to 10 to 25 to 50 and no effect on the execution plan. Anyone have any suggestions?

I encountered the following error:

named pipes provider: could not open a connection to sql server 53.

this error was raised for some 15 minutes and it seemed that the database operation was halted during that time.

surprisingly, this error was written in the application error log table through same connection in the same sql server.

i checked the sql server error log but no error was logged.

i could not figure out. please help.

Hi Team,

Can you please help to understand the tempdb issue. we have a server and using Seperate SSD drive for tempdb

we had a Index rebuild activity on SQL server 2016 Enterprise edition ater the operation Tempdb drives gradullay increased 

and never used the sort_in tempdb option equal to on. How to troubleshoot the issue?

Regards,

Nasar.

Hi,

I am running query using sys.dm_exec_procedure_stats to get the stored procedure usage report and little confused that it shows latest time like today but Execution count is NULL so what i should understand?

Is it Not using Stored Procedure at all since Last Sql server Restarted?

How I should interpreting Last_Execution_Time and execution_count columns as 4/1/2020 NULL
I was reading definitions but h=got confused:

Last_execution_time Last time at which the plan started executing
last time the execution plan was used to execute a query
execution_count Number of times that the plan has been executed since it was last compiled
number of times the plan has been used to execute a query

Thanks

hi, years ago when i did my last backup i think there were some rules about the backup going to a different drive than where the db resides and restores coming from a different drive.  

i just backed up my 112 gig db from my c: drive to my e: drive.  i'll find out what my e: drive really is but "explorer" says its local and "storage" only knows about my c: drive.

if i need to do a restore, is there anything i can mitigate now? will i be able to run a restore from a different drive?  

How to handle DB locks in SQL Server 2016.

Like kind of alerts or some solution for automatically release locks after some threshold?

Please help me.

When I try create a database I get:

Msg 1801, Level 16, State 3, Line 1
Database 'XXXX' already exists. Choose a different database name.

I then try drop the database but get:

Msg 3701, Level 11, State 11, Line 8
Cannot drop the database 'XXXX', because it does not exist or you do not have permission.

The database did exist previously. It's not currently listed in sys.databases.
Microsoft SQL Server 2017 (RTM-CU16) (KB4508218) - 14.0.3223.3 (X64)

I've had this before and was able to create the database after restarting SQL but I cannot keep restarting the server.

Hello,

We have a SQL Server 2012 VM running in our DEV environment.  One of our sysadmins created a backup of this VM and restored as "DEV2" just to test out the DR process.  I logged onto to DEV2 and everything seemed to be fine.  One thing I observed was some of the sql agent jobs started to run (as scheduled) on DEV2.  So I used tsql to disabled all of the jobs.  However I noticed I was getting job failure notifications via email which I thought was odd since all the jobs had already been disabled and I confirmed it this.  When I view the job log history, indeed I saw recent dates and they in fact failed.  My question is: why these jobs continue to run on DEV2?  

When I ran these tsql's on DEV2:

select @@SERVERNAME     -> output "DEV" is the original DEV sql server

SELECT CONVERT(sysname, SERVERPROPERTY('servername'));    -> output "DEV2"

Obvisously there's a discrepancy here.   Does this have something to do with why agent jobs ran on DEV2?

Thanks