Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
Home
Discussion Groups
DB Engine
SQL ServerMSDESQL Server CE
Services
Analysis (Data Mining)Analysis (OLAP)DTSIntegration ServicesNotification ServicesReporting Services
Programming
CLRConnectivitySQLXML
Other Technologies
ClusteringEnglish QueryFull-Text SearchReplicationService Broker
General
Data WarehousingPerformanceSecuritySetupSQL Server ToolsOther SQL Server Topics
DirectoryUser Groups
Related Topics
MS AccessOther DB ProductsMS Server Products.NET DevelopmentVB DevelopmentJava DevelopmentMore Topics ...

SQL Server Forum / Other Technologies / Full-Text Search / October 2004

Tip: Looking for answers? Try searching our database.

Removing duplicate dat

Thread view: 
Enable EMail Alerts  Start New Thread
Thread rating: 
Paul - 25 Oct 2004 20:21 GMT
Hi

I have a 1.5 million row table with a text field.  Can
anybody think of a way to remove any duplicate rows where the text field
contains the same data (and a
way which will not take days to run!)

For example  if I have two rows with "SQL SERVER IS GREAT", I want to remove
one of them.

Thanks
Hilary Cotter - 26 Oct 2004 03:12 GMT
here is an example of something that works for me:

create table textstuff
(pk int not null primary key,
textcol text)
go

--insert statements

declare @int int
select @int=max(datalength(textcol)) from textstuff

select pk, checksum=checksum(substring(textcol,1, @int)) into holding from
textstuff order by 2

select pk, holding.checksum from holding,
(select checksum, test=count(checksum) from holding group by checksum having
count(checksum) >1) as a
where holding.checksum=a.checksum

rows which have identical checksums will show up here.

Signature

Hilary Cotter
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html

> Hi
>
[quoted text clipped - 7 lines]
>
> Thanks
Paul - 26 Oct 2004 14:47 GMT
That's awsome, thanks Hilary

> here is an example of something that works for me:
>
[quoted text clipped - 30 lines]
> >
> > Thanks
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2009 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.