Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
Home
Discussion Groups
DB Engine
SQL ServerMSDESQL Server CE
Services
Analysis (Data Mining)Analysis (OLAP)DTSIntegration ServicesNotification ServicesReporting Services
Programming
CLRConnectivitySQLXML
Other Technologies
ClusteringEnglish QueryFull-Text SearchReplicationService Broker
General
Data WarehousingPerformanceSecuritySetupSQL Server ToolsOther SQL Server Topics
DirectoryUser Groups
Related Topics
MS AccessOther DB ProductsMS Server Products.NET DevelopmentVB DevelopmentJava DevelopmentMore Topics ...

SQL Server Forum / Other Technologies / Full-Text Search / September 2007

Tip: Looking for answers? Try searching our database.

Integration of Lucene.Net and SQL Server 2005?

Thread view: 
Enable EMail Alerts  Start New Thread
Thread rating: 
DC - 11 Sep 2007 12:07 GMT
Hi,

I am looking for an alternative to the SQL Server Fulltext Engine
since it is just not fast enough and does not offer the sorting I
require. Also, my customer wants a "did you mean google" when "gogle"
was entered feature. From what I have seen all that is possible with
Lucene.Net.

I am planning to export a table of about 3 Million keywords to
Lucene.Net and of course I have to implement a way to keep that index
up to date (a latency of one hour is allowed). Also, I would still
like to initiate searching through stored procs. Ideally a table-value
function with CLR integration would get a list of matching ID's from
Lucene.

I wanted to check if maybe someone has used such approach or heard
about something similar, or if there are general restrictions that
make the success of this approach unlikely.

TIA for any comments,

Regards
DC
Hilary Cotter - 11 Sep 2007 15:12 GMT
I did this. It works quite well. SQL FTS is faster at indexing and you have
to jump through hoops to get Lucene working correctly, but properly done it
is an alternative, especially when you need to do filtering.

Signature

RelevantNoise.com - dedicated to mining blogs for business intelligence.

Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html

Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com

> Hi,
>
[quoted text clipped - 19 lines]
> Regards
> DC
DC - 11 Sep 2007 16:46 GMT
> I did this. It works quite well. SQL FTS is faster at indexing and you have
> to jump through hoops to get Lucene working correctly, but properly done it
[quoted text clipped - 30 lines]
>
> - Zitierten Text anzeigen -

Thank you for letting my know, Hilary! That's encouraging. Do you
consider designing an "SQL Server / Lucene.Net Bridge" product?
Hilary Cotter - 11 Sep 2007 18:22 GMT
I am writing an article on it for simple-talk.

Signature

RelevantNoise.com - dedicated to mining blogs for business intelligence.

Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html

Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com

>> I did this. It works quite well. SQL FTS is faster at indexing and you
>> have
[quoted text clipped - 38 lines]
> Thank you for letting my know, Hilary! That's encouraging. Do you
> consider designing an "SQL Server / Lucene.Net Bridge" product?
DC - 12 Sep 2007 08:17 GMT
> I am writing an article on it for simple-talk.
>
[quoted text clipped - 50 lines]
>
> - Zitierten Text anzeigen -

Thank you! I will wait for that article before I do anything.

Regards
DC
Hilary Cotter - 12 Sep 2007 13:45 GMT
Here is the code. The field I am indexing and searching is Contents.

using System.Data;
using System.Data.SqlClient;
using System.Data.SqlTypes;
using System.Collections;
using Microsoft.SqlServer.Server;
using Lucene.Net;
using Lucene.Net.Search;
using Lucene.Net.Documents;
using Lucene.Net.QueryParsers;
using Lucene.Net.Analysis;
using Lucene.Net.Analysis.Standard;

public partial class UserDefinedFunctions
{

   [Microsoft.SqlServer.Server.SqlFunction(DataAccess =
DataAccessKind.Read,
   FillRowMethodName = "FillRow", SystemDataAccess =
SystemDataAccessKind.Read, TableDefinition = "PK int")]
   public static IEnumerable LuceneSearch(string SearchPhrase)
   {
       Searchable[] remoteSearch = new Searchable[7];
       remoteSearch[0] = new IndexSearcher("//10.0.0.1/14");
       remoteSearch[1] = new IndexSearcher("//10.0.0.2/15");
       remoteSearch[2] = new IndexSearcher("//10.0.0.3/16");
       remoteSearch[3] = new IndexSearcher("//10.0.0.4/17");
       remoteSearch[4] = new IndexSearcher("//10.0.0.4/18");
       remoteSearch[5] = new IndexSearcher("//10.0.0.4/18a");
       remoteSearch[6] = new IndexSearcher("//10.0.0.4/19");
       ParallelMultiSearcher searcher = new
ParallelMultiSearcher(remoteSearch);

       QueryParser qp = new QueryParser("Contents", new
Lucene.Net.Analysis.Standard.StandardAnalyzer());
       Query query = qp.Parse(SearchPhrase);

       Hits hits = searcher.Search(query);

       int[] PKs = new int[hits.Length()];
       for (int i = 0; i < hits.Length(); i++)
       {
         Document doc = hits.Doc(i);
         PKs[i] = int.Parse(doc.Get("PK"));
       }

       return postids ;
   }

   public static void FillRow(object row, out int PK)
   {
       PK= (int)row ;
   }

};

Signature

RelevantNoise.com - dedicated to mining blogs for business intelligence.

Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html

Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com

>> I am writing an article on it for simple-talk.
>>
[quoted text clipped - 64 lines]
> Regards
> DC
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2009 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.