A new search engine in DotNetAge2

• Introduction to OpenSearch
  • OpenSearch description document
  • Autodiscovery
• DNA OpenSearch library
  • OpenSearch description document
  • Implement OpenSearch provider
• DotNetAge OpenSearch engine
• How to create search engine for DotNetAge

Introduction to OpenSearch

---

"OpenSearch is a collection of simple formats for the sharing of search results. " - OpenSearch.org


OpenSearch was originally developed at Amazon.com's A9 incubator. It's a specification under the Creative Commons Attribution-ShareAlike License, covering discovery and description documents for search engines, expression of queries, and the convention of RSS 2.0 or Atom Web feed results. It is very RESTful in nature and complementary to the Atom Publishing Protocol (APP).


In fact, many have called for OpenSearch to serve as the query aspect of APP, which provides a way to access
identified or located results, but no mechanism for ad hoc query. With all this affinity to Atom and REST, OpenSearch is a natural topic for this Agile Web column. OpenSearch 1.0 is still the latest full version; it has been around since 2005.


I'll be touching upon the the following topics:

• Create an OpenSearch description document for your search engine with DNA.OpenAPI.OpenSearch library
• Adding autodiscovery links to your web site so browsers/clients can easily find your OpenSearch description document.
• Introduces DotNetAge search engine
• Create your search engine and extend your DotNetAge

OpenSearch description document

---

The first step in making your site's search engine accessible via OpenSearch is to create what's called an OpenSearch description document. An OpenSearch description document is simply an XML file that describes your search's interface. There are many optional elements which you can add if you want to, but in its basic form an OpenSearch description document looks something like this:

<OpenSearchDescription>
   <Developer>dotnetage.com development teamDeveloper>
   <ShortName>dotnetage.comShortName>
   <Description>Search DotNetAge contentsDescription>
   <SyndicationRight>openSyndicationRight>
   <Url template="https://dotnetage.com/Search?terms={searchTerms}" type="text/html" rel="results" />
   <Url template="https://dotnetage.com//OpenAPI/Osd" type="application/opensearchdescription+xml" rel="self"/>
   <Image height="16" width="16" type="image/x-icon">https://dotnetage.com/Content/Images/dna.icoImage>
OpenSearchDescription>

Most of the elements are pretty self-explanatory and when creating your own OpenSearch description document you should obviously replace the sample values I've included above with values which more accurately reflect the characteristics of the search to which the document points.The two elements which do warrant a brief explanation are Url and Query.The Url element is used to describe an interface which a client can use to request search results. If you're simply trying to
allow users to search your site via OpenSearch, then you'll only need a Url element with a type attribute of "text/html" as shown in the example above.




Note: Although most of you won't need to include more then one, it is perfectly acceptable for a OpenSearch description document to contain multiple Url elements. For example, if search results are also available as XML, RSS, or Atom, you would create a Url element for each and in the type attribute you would include the MIME type which indicates the format in which the search results are returned.




The Url element's template attribute is where the real magic happens. This is where you provide the URL and format that search clients should use to query your search engine. OpenSearch 1.1 defines a number of parameters which you can use as placeholders in the template. The most important of these is the "searchTerms" parameter which is replaced by the keyword(s) for which the client is searching. In the example above, I've also included the "startIndex" parameter as an optional template parameter. You indicate that the parameter is optional by adding the "?" character to the end of the parameter name. The search client will use an empty string as a value if no value is specified for an optional parameter.


The Query element is used to define a search that a client can perform. In our scenario it's not all that useful, but in more complex applications you can use the Query element to pass parameters (ie. spelling corrections, number of search results, or related search terms) back to the client. The Query element in our sample description document has the role of "example" which indicates that it's simply included to provide search clients with a sample query with which to test the search engine. Whatever example terms you provide are expected to return search results.That should be all the information you need in order to create an OpenSearch description document for your site's search engine, but if you find you need any more information the best place to turn is to the OpenSearch description document section of the OpenSearch 1.1 specification.

Autodiscovery

---

Some browsers (notably Firefox 2.0/3.0/3.5 and IE 7/8) support OpenSearch autodiscovery links and allow the search engine to be added to their search boxes. This is as simple as opening up any page with an OpenSearch autodiscovery link and then clicking on the search box icon and noting the new menu option 'Add "dotnetage.com"'. Clicking on that link will add dotnetage.com search to the list of managed search engines.







We could add the link tag inside the page head tag to let IE / Firefox find our OpenSearch description document. The html code like below:

<html>
<head>
<link rel="search" 
          type="application/opensearchdescription+xml" 
          title="DotNetAge" 
          href="https://dotnetage.com/OpenAPI/Osd" />
...
head>
...
html>

DNA OpenSearch library

---

DNA OpenSearch is an opensource library that makes easy to implement the OpenSearch 1.1 specification for .NET web applications.

OpenSearch description document

---

DNA OpenSearch lib not only defines the object model for OpenSearch1.1 and also provides a very easy way to create OpenSearch description document. You could use the OsdBuilder to build your OpenSearch description document without reading OpenSearch specification.


Now let's take a look at a OpenSearch description document generate sample for ASP.NET Mvc:

public ActionResult Osd()
{
     var web = WebSite.Open();
     //Create the OsdBuilder instance 
     var osdBuilder = new OsdBuilder("dotnetage.com","Search DotNetAge contents");

      //Add icon file as search engine icon
      osdBuilder.AddIcon(web.ShortcutIconUrl);
      string baseUrl = Request.Url.Scheme + "://" + Request.Url.Authority;

      //Add the search url
      osdBuilder.AddSearchUrl(baseUrl + "/Search?terms={searchTerms}", "text/html");

      //Add the plugin url for Firefox and IE
      osdBuilder.EnableClientPlugin(baseUrl + "/" + Url.Action("Osd", "OpenAPI", new { Area = "" }));
       
       //Return the OpenSearch description document conent.
       return Content(osdBuilder.GetXml(), "text/xml");
}

Output:

<OpenSearchDescription>
   <ShortName>dotnetage.comShortName>
   <Description>Search DotNetAge contentsDescription>
   <Url template="https://dotnetage.com/Search?terms={searchTerms}" type="text/html" rel="results" />
   <Url template="https://dotnetage.com//OpenAPI/Osd" type="application/opensearchdescription+xml" rel="self"/>
   <Image height="16" width="16" type="image/x-icon">https://dotnetage.com/Content/Images/dna.icoImage>
OpenSearchDescription>

Implement OpenSearch provider

---

DNA OpenSearch lib provides a simple interface to search the content, developers can implement this interface to process OpenSearch search query and ouput Atom or Rss objects/xml file as search results.

public interface IOpenSearchProvider
{
   string Title { get;  }

   void Search(ref OpenSearchQuery query, out OpenSearchAtomResult result);

   void Search(ref OpenSearchQuery query,out OpenSearchRssResult result);
}

The OpenSearchRssResult class is inhert from RssChannel class that is reference from DNA.OpenAPI.RSS.dll.(You can get this library in DotNetAge source code project).

[XmlRoot("channel"), Serializable]
public class OpenSearchRssResult : RssChannel, IOpenSearchResult
{
        [XmlElement("Query", Namespace = "http://a9.com/-/spec/opensearch/1.1/")]
        public OpenSearchQuery Query { get; set; }

        [XmlElement("link", Namespace = "http://www.w3.org/2005/Atom")]
        public AtomLink SearchLink { get; set; }
}

The OpenSearchAtomResult class is inhert from AtomFeed class that is contains in DNA.OpenAPI.Atom.dll.

[XmlRoot("feed", Namespace = "http://www.w3.org/2005/Atom"), Serializable]
public class OpenSearchAtomResult:AtomFeed,IOpenSearchResult
{
        [XmlElement("Query", Namespace = "http://a9.com/-/spec/opensearch/1.1/")]
        public OpenSearchQuery Query { get; set; }

        [XmlElement("totalResults", Namespace = "http://a9.com/-/spec/opensearch/1.1/")]
        public int TotalResults { get; set; }

        [XmlElement("startIndex", Namespace = "http://a9.com/-/spec/opensearch/1.1/")]
        public int StartIndex { get; set; }

        [XmlElement("itemsPerPage", Namespace = "http://a9.com/-/spec/opensearch/1.1/")]
        public int ItemsPerPage { get; set; }
}

Developers could be inhert the OpenSearchProvider base class to impletement the OpenSearch engine.

    public abstract class OpenSearchProvider:IOpenSearchProvider
    {
        public string Name { get; set; }

        public abstract string Title { get; }

        public abstract IEnumerable GetAtomResult(string searchTerms, int pageIndex, int pageSize,out int 

totalRecords);
        
        public abstract IEnumerable GetRssResult(string searchTerms, int pageIndex, int pageSize, out int 

totalRecords);

        void IOpenSearchProvider.Search(ref OpenSearchQuery query, out OpenSearchAtomResult result)
        {
            var pageSize = query.Count > 0 ? query.Count : 20;
            int total = 0;
            var resultSet = GetAtomResult(query.SearchTerms, query.StartIndex, pageSize, out total);

            query.TotalResults = total;

            var feed = new OpenSearchAtomResult()
            {
                Entries = resultSet.ToList(),
                TotalResults = total,
                Query = query,
                StartIndex = query.StartIndex,
                ItemsPerPage = pageSize,
                Title = query.Title
            };
            result = feed;
        }

        void IOpenSearchProvider.Search(ref OpenSearchQuery query, out OpenSearchRssResult result)
        {
            var pageSize = query.Count > 0 ? query.Count : 20;
            int total = 0;
            var resultSet = GetRssResult(query.SearchTerms, query.StartIndex, pageSize, out total);
            query.TotalResults = total;

            var feed = new OpenSearchRssResult()
            {
                Items = resultSet.ToList(),
                Query = query,
                Title = query.Title
            };
            result = feed;
        }
    }

The OpenSearchProvider allows developer to create a search engine that support paging search result and returns Atom / Rss feeds.You just only override the two get result methods:

public abstract IEnumerable GetAtomResult(string searchTerms, int pageIndex, int pageSize,out int totalRecords);
        
public abstract IEnumerable GetRssResult(string searchTerms, int pageIndex, int pageSize, out int totalRecords);

In next section i will show you how to create a OpenSearchProvider and how to use it.

DotNetAge OpenSearch engine

---

There is a new search engine that base on OpenSearch add to DotNetAge2. There are the search engine design goals below:

• Support provider base search source.
• Auto generate OpenSearch description document.
• Auto generate autodiscovery links in page head tag.
• Search result page.

You can try the site search (On the top of the right corner) on dotnetage.com to test new search engine.Also you can use IE7/8 or FireFox to access dotnetage.com and input the search keywords in browser search box.

How to create search engine for DotNetAge

---

The DotNetAge search engine is base on OpenSearch so we just only add a new class that inhert from OpenSearchProvider and add the provider type in Web.config. The search engine will send the request to all registered search providers and display the search result in respective tabs.


DotNetAge only use the AtomFeed as search result in other words you could only overide the GetAtomResult method in your provider class.The example code below i have implemented a file search provider

namespace DNA.Example
{
    public class FileSearchProvider : OpenSearchProvider
    {
        public override string Title
        {
            get { return "Files"; }
        }

        public override IEnumerable GetAtomResult(string searchTerms, int pageIndex, int pageSize, out int 

totalRecords)
        {
            var searchPath=HttpContext.Current.Server.MapPath("~/YourSearchPath");

            var files= Directory.GetFiles(searchPath)
                                .Where(name=>name.Contains(searchTerms))
                                .Skip((pageIndex-1)*pageSize)
                                .Take(pageSize);

            var entries = new List();
            totalRecords = Directory.GetFiles(searchPath).Length;

            foreach (var file in files)
            { 
                var fileInfo=new FileInfo(file);
                var fileUrl="/YourSearchPath/"+Path.GetFileName(file);
                var builder = new AtomEntryBuilder(Path.GetFileNameWithoutExtension(file),fileUrl );
                builder.SetPublished(fileInfo.CreationTime);
                builder.SetContent(fileInfo.Name, src: fileUrl);
                builder.SetLastUpdated(file.LastWriteTime);
                builder.AddLink(fileUrl);
                entries.Add(builder.Entry);
            }
            return entries;
        }

        public override IEnumerable GetRssResult(string searchTerms, int pageIndex, int pageSize, out int 

totalRecords)
        {
           return null;
        }
    }
}

Now we need to register the FileSearchProvider to DotNetAge web.config in dna/searchProviders section.

  <dna>
    <services>
      <add name="pages" type="DNA.Mvc.DynamicUI.DynamicPageService,DNA.Mvc" />
      <add name="widgets" type="DNA.Mvc.DynamicUI.WidgetUIServcie,DNA.Mvc" />
      <add name="security" type="DNA.Mvc.Security.SecurityActionService,DNA.Mvc" />
      <add name="files" type="DNA.Mvc.FileWebResourceService,DNA.Mvc" />
      <add name="authentication" type="DNA.Mvc.Security.FormsAuthenticationService,DNA.Mvc" />
      <add name="membership" type="DNA.Mvc.Security.AccountMembershipService,DNA.Mvc" />
      <add name="logging" type="DNA.Mvc.Logging.LogService,DNA.Mvc" />
      <add name="Publishing" type="DNA.Mvc.Areas.Publishing.Models.PublishingService,DNA.Mvc" />
    services>
    <searchProviders>
      <add name="sites" type="DNA.Mvc.DynamicUI.WebPageSearchProvider,DNA.Mvc" />
      <add name="articles" type="DNA.Mvc.Areas.Publishing.ArticleSearchProvider,DNA.Mvc" />
      <add name="forums" type="DNA.Mvc.Areas.Community.PostSearchProvider,DNA.Mvc" />
      <add name="files" type="DNA.Example.FileSearchProvider,DNA.Example" />   
    searchProviders>
  dna>

When search on your DotNetAge you see a tab name "files" in search result page that contains the file search result links.

 

---

• Average:
• Reads
  (2272)
• Permalink