Search string including a forward slash doesn't work :(

Thread view:

Enable EMail Alerts

Start New Thread

Thread rating:

Andy Smith - 24 Nov 2004 18:11 GMT

I'm searching a column which holds XML data in a varchar field.
When I search on reference 'XX12345678' I get results, but when I search on
'XX12345678/1' (the full reference number for my application) I get no
results.

eg.

SELECT *
FROM FullTextIndexedTable
WHERE CONTAINS(metadata,@searchpattern)

When @searchpattern = 'XX12345678' then I get a match, when it's
'XX12345678/1' then I don't!

The XML source data definitely contains the string 'XX12345678/1' several
times, but that match is not found.

Can any one tell me why including a forward slash (/) stops the match from
being found? Is this a bug? Is there a workaround/fix?

Many thanks,
Andy.

ps. Server is Win2k3 and @@Version is
"Microsoft SQL Server 2000 - 8.00.760 (Intel X86) Dec 17 2002 14:22:05
Copyright (c) 1988-2003 Microsoft Corporation Standard Edition on Windows NT
5.2 (Build 3790: ) "

Reply to this Message

Hilary Cotter - 24 Nov 2004 20:29 GMT

The word breaker for Win2003 does strange things when encountering some
formatting marks, like <, and > which means that it doesn't index words
between and sometimes words touching these characters. The / mark seems to
also fit in this category. Perhaps change your content and searches to be on
something like XX12345678_1

Signature

Hilary Cotter
Looking for a SQL Server replication book?
Now available for purchase at:
http://www.nwsu.com/0974973602.html

> I'm searching a column which holds XML data in a varchar field.
> When I search on reference 'XX12345678' I get results, but when I search
[quoted text clipped - 25 lines]
> NT
> 5.2 (Build 3790: ) "

Reply to this Message

John Kane - 25 Nov 2004 05:00 GMT

Andy,
First of all, thank you for providing the @@version info as the OS platform
version is most important in understanding this issue!
The reason is the OS-supplied word breaker that ships with Win2003 & WinXP -
langbrk.dll - treats punctuation character s such as "/" (forward slash)
differently than the Win2K OS-supplied word breaker - infosoft.dll. This
issue has been discussed many times, see:
http://groups.google.com/groups?q=langwrbk+infosoft.

This is an un-documented bug: Shiloh bug# 351310 "Full Text Search Win2K
word breaker does not ignore punctuation unless separated by white space".

Specifically on Win2003 and using langwbrk.dll, the string "'XX12345678/1'"
is broken into tokens:

Original text: 'XX12345678/1'
IWordSink::PutWord: cwcSrcLen 10, cwcSrcPos 0, cwc 10, 'XX12345678'
IWordSink::PutAltWord: cwcSrcLen 1, cwcSrcPos 11, cwc 1, '1'
IWordSink::PutWord: cwcSrcLen 1, cwcSrcPos 11, cwc 3, 'NN1'

While the same search string on Win2K would succeed becasue of the above
bug. The workaround for this bug on Win2K is to use the Neutral "Language
for Word Breaker" when you define the column using the FT Indexing Wizard.

Regards,
John

> I'm searching a column which holds XML data in a varchar field.
> When I search on reference 'XX12345678' I get results, but when I search on
[quoted text clipped - 23 lines]
> Copyright (c) 1988-2003 Microsoft Corporation Standard Edition on Windows NT
> 5.2 (Build 3790: ) "

Reply to this Message

Andy Smith - 25 Nov 2004 09:49 GMT

Thanks John and Hilary. I realised after a bit more digging on the groups
that I was not alone in this problem!

John, you said at the bottom of the post was "The workaround for this bug
on Win2K is to use the Neutral "Language for Word Breaker"

Did you mean Win2k3? I thought this wasn't a problem on Win2k?

Many thanks again,
Andy.

> Andy,
> First of all, thank you for providing the @@version info as the OS platform
[quoted text clipped - 52 lines]
> NT
> > 5.2 (Build 3790: ) "

Reply to this Message