Skip to main content

Data purging and Archival from VLDB

How to delete Records from a Very Large Tables ?

If we have to delete a large records based on some conditions like year wise - month wise from very large table and our SLA is without degrading system performance then we will come into problem .We can't use TRUNCATE TABLE and we may run out of disk space and then DELETE query fails and nothing is deleted.

The Real-Problem (from SQL Bible)

A DELETE executes as a transaction where it will not commit until the last record is deleted. DELETE physically removes rows one at a time and records each deleted row in the transaction log. If the number of records in a table is small, a straight out DELETE is fine, however, with a large table the DELETE will cause the transaction log to grow. When this happens the system IO performance is degraded. Also, the table will be locked which will affect the application. The person executing the DELETE query will usually panic because what seems to be a simple task takes a long time and there is no visible progress from Query Analyzer. If the transaction log runs out of disk space then the DELETE fails and the transaction rolls back and nothing ends up deleted.

A Solution:
A solution is to use the combination of a WHILE loop and @@ROWCOUNT to delete the records in manageable blocks
Below is a script that you can use to do bulk deletions.

select count(*) from prps_delete where left(erdat,4)='2012'


WHILE 1 = 1
      DELETE prps_delete WHERE left(erdat,4)='2012'
      IF @@ROWCOUNT = 0

(10000 row(s) affected)

(10000 row(s) affected)

(10000 row(s) affected)

(10000 row(s) affected)


(526 row(s) affected)
select count(*) from prps_delete where left(erdat,4)='2012'



Popular posts from this blog

mongoDB error : aborting after fassert() failure

What to do when facing errors on mongoDB “aborting after fassert() failure” I like errors, in mongoDB this is the first error I faced and luckily many times. This error i faced during restoring name-space on local and restarting db system. I am still searching the exact root cause of this issue but i am able to resolve the current problem through below steps. Remove all relevant namespace files from data-file route path.. Now repair mongo instance using mongod process. mongod --repair ////////// execute command from bin folder path  Then start server using mongd process, if started server successfully then .. mongod  ////////// execute command from bin folder path Restore last backups as normal process. Now check database by connecting mongo shell. Thanks for reading,  Please comment your experience if you faced and also share knowledge if you have better steps to resolve...  

SQL71562: external references are not supported when creating a package from this platform

Last week I got this error from one of developer who was trying to deploy his project from Testing server to SQL Azure QA server. He was using “Deploy Database to SQL Azure” option from SSMS Tool-Task option. After connecting to SQL Azure portal when operation started to deployment below errors occurs. Validation of the schema model for data package failed. Error SQL71562: Error validating element has an unresolved refrence to object xx.dbo.xxxx external refrences are not supported when creating a package from this platform . Reason: The reason of the this error was; some functions of project was dependent on master database and only single database was being deploy to SQL Azure. DACFx must block Export when object definitions (views, procedures, etc.) contain external references, as Azure SQL Database does not allow cross-database external references So, this error was coming. Solution : I suggested him to create those function to locally

How to add an article in Transactional Replication

If we have a set-up of Transactional Replication for Data Distribution running and wanting to add new object to replication on other server we can follow below process. To add an article In Transaction replication with PUSH Subscription