Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
Home
Discussion Groups
DB Engine
SQL ServerMSDESQL Server CE
Services
Analysis (Data Mining)Analysis (OLAP)DTSIntegration ServicesNotification ServicesReporting Services
Programming
CLRConnectivitySQLXML
Other Technologies
ClusteringEnglish QueryFull-Text SearchReplicationService Broker
General
Data WarehousingPerformanceSecuritySetupSQL Server ToolsOther SQL Server Topics
DirectoryUser Groups
Related Topics
MS AccessOther DB ProductsMS Server Products.NET DevelopmentVB DevelopmentJava DevelopmentMore Topics ...

SQL Server Forum / Other Technologies / Full-Text Search / January 2007

Tip: Looking for answers? Try searching our database.

good, thorough documentation on full text issues?

Thread view: 
Enable EMail Alerts  Start New Thread
Thread rating: 
tbh - 18 Jan 2007 12:13 GMT
hi,

out website is starting to get serious about switching from SQL 2000 to SQL
2005. in the process we hope to address a few weak points in our site's
fulltext searching.

i'd like to find the best possible documentation or courses to get up to
speed on this. i have googled quite extensively and found some detailed
information, e.g. at msdn, in newsgroups, and the two fulltext articles
here:
  http://www.simple-talk.com/author/hilary-cotter/

(which strike me as very good as far as they go.)

my question is whether there are good books, articles, or courses that go
deeper still. maybe someone has assembled a list that i haven't found yet?

in particular i'm interested in the following issues:
- i've read that SQL 2005 promises improvements in fulltext search, but i
haven't seen anything dramatic in my reading and testing yet [*]; (one
example: supposedly the german noise word list was improved, but my
impression is it's the same as before; this is no big deal, we will replace
it with something like the 100 most frequent German words when we switch to
2005 -- good breaking point)
- i'd like to learn more about language-specific issues such as how words
are parsed and stemming is done, in particular how can i tune and test these

i'd be grateful for any pointers.

cheers,

Tim Hanson

[*] one quote from one of the above-sited articles is intriguing:

  Similarly a table can be full-text indexed in a single
  catalog; or to put it another way - a table's full-text
  index cannot span catalogs. However, in SQL 2005 you
  can full-text index views which may reside in different
  catalogs than the underlying base tables. This provides
  performance benefits and allows partitioning of tables.

however i'd need to learn much more to decide whether i can benefit from it.
Hilary Cotter - 18 Jan 2007 13:10 GMT
I did something for searchsql.target.com on SQL FTS which IMHO is pretty
through. I am not sure if it has been posted yet:(

There is a problem with full-text indexed views which has surfaced recently
which does not offer the best performance. I would advise you to test.

I think we have the largest implementation of SQL FTS - we are indexing over
1.5 terrabytes.

Signature

Hilary Cotter

Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html

Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com

> hi,
>
[quoted text clipped - 41 lines]
> however i'd need to learn much more to decide whether i can benefit from
> it.
tbh - 18 Jan 2007 13:48 GMT
thanks.

>I did something for searchsql.target.com on SQL FTS which IMHO is pretty
>through. I am not sure if it has been posted yet:(

hmmm. i don't find that URL. http://searchsql.target.com is there something
missing?

> There is a problem with full-text indexed views which has surfaced
> recently which does not offer the best performance. I would advise you to
> test.

good to know, thanks.

> I think we have the largest implementation of SQL FTS - we are indexing
> over 1.5 terrabytes.

also good to know. our needs are considerably smaller, but it would still be
great if all searches were quicker. (they tend to be slower the more hits
there are -- which of course makes sense.) if there is some trick (e.g.
using partitioning) that could help i'd be all ears and eyes. :)
Mike C# - 23 Jan 2007 16:42 GMT
> thanks.
>
[quoted text clipped - 3 lines]
> hmmm. i don't find that URL. http://searchsql.target.com is there
> something missing?

Maybe he meant http://searchsqlserver.techtarget.com/?  I don't think Target
Department stores has a public website for SQL Server articles...  I could
be wrong though...
Simon Sabin - 18 Jan 2007 20:29 GMT
Hello tbh,

With full text on sql 2005, you need to make sure you manage resources properly.
We initially just upgraded and ran a reindex. However it didn't run much
faster. However this was becasue our DB is larger than our server memory.
SQL was contending with full text for resources. Fixing SQL memory resolved
the issue allowing FT to operate efficiently. This resulted in huge improvements.

We have upgraded from quad 32 bit boxes to 2 xdual core 64 bit boxes and
the performance has rocketed.

The biggest is that under load FT doesn't complain, no more errors saying
FT couldn't find a row or one of the many other errors due to searching data
during changes being indexed.

Bottom line is go for it.

One word or warning, make sure you update stats especially on date columns.
in 2005 the optimiser does get estimated rowcounts from FT however this can
cause it to put a FT query with a nested loop in a query plan. This can be
very painful if the optimiser has got the wrong row count due to out of date
statistics.

Simon Sabin
SQL Server MVP
http://sqlblogcasts.com/blogs/simons

> hi,
>
[quoted text clipped - 46 lines]
> however i'd need to learn much more to decide whether i can benefit
> from it.
tbh - 19 Jan 2007 13:26 GMT
thanks, Simon.

we have been thinking about 64 bit as a server upgrade is likely to be part
of the project. haven't heard numbers about price increases compared with
performance gains. i take it you feel 64bit is worth the higher price tag?

cheers,

Tim

p.s. back to my question, can you recommend any good books or literature??
:)
Simon Sabin - 19 Jan 2007 20:54 GMT
Hello tbh,

Good books on full text don't exist, best bet is to google. You will probably
end up at one of Hilarys articles, of sometimes mine.

As for 64 bit, I think the cost is negligible.

I have just had a quote for a 4x dual core 64 opteron with 32Gb of memory
and 900Gb of local storage all for £12k, with fibre channel connectivity.

Rediculous price. Compare that with the cost of resource, its now so much
easier to through hardware at a problem than spend time and money on resource.

Simon Sabin
SQL Server MVP
http://sqlblogcasts.com/blogs/simons

> thanks, Simon.
>
[quoted text clipped - 9 lines]
> p.s. back to my question, can you recommend any good books or
> literature?? :)
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2009 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.