Enterprise search software may hold the key
The evolution of enterprise search has been approaching, but never quite reaching, the level of accuracy and usability that many organizations feel is really needed.
More than a decade of big promises with mostly unfulfilling results has characterized the market. However, the recent infusion of ML (Machine Learning) and AI (Artificial Intelligence) into enterprise search software is an effort by leading vendors to change that.
It’s difficult to compare the vendors of these solutions in a true side by side, because each product tends to address different aspects of enterprise search and with different strategies. However, we can do a rundown of some of the best features across the diverse range of search approaches.
IBM Watson Discovery
Big Blue’s approach to search in the enterprise has been to address the vast quantities of dark data building up in organizations — content such as documents, Excel spreadsheets, PowerPoint presentations, emails and other formats that aren’t accessible by metadata-based search because they aren’t tagged.
Watson Discovery looks to solve the problem with AI-driven NLP (Natural Language Processing), reading files and presenting them to the user based upon more conversational-language queries. It is a tool set available in both cloud and on-premises versions that businesses can integrate into other applications. Users can train the Discovery AI what is useful and what isn’t, making it not only more efficient at search but also able to learn users’ preferences. A multi-label classifier is built in, so the user is able to “teach” Discovery what to look for. Organizations can also provide an indexing system that Discovery can likewise learn and use autonomously.
Watson Discovery includes document connectors for Box, Salesforce, SharePoint and other platforms. With IBM’s Cloud Pak for Data Integration, Discovery can run on any of the major public cloud environments as well as private clouds.
Salesforce Einstein Search
Augmenting Salesforce Global Search, the Einstein search engine has improved the CRM giant’s enterprise offering by applying ML to better dial-in search’s context sensitivity within the platform. Introduced in September 2019, Einstein Search identifies relevant user-specific data points that enable it to nominally self-train on the user’s intent. As users issue queries, Einstein becomes increasingly sensitive to what the user is really going for.
Einstein Search can try to further narrow search results by learning a user’s operational boundaries and focusing the search within those borders — such as specific industries or geographical territories, for instance. The system doesn’t discard other search results, but the AI moves the ones it believes to be more relevant to the top of the list, thus decreasing the time for a user to access the information they were most looking for.
Microsoft’s enterprise search software unites the AI technology already embedded in their Bing web search with the personalized insights engine of the Graph. Microsoft’s approach to improving its search product has been typically incremental, with a particularly prominent announcement at Ignite 2019 –Project Cortex — the much-anticipated knowledge network that is slated for initial release in summer 2020.
Features announced include attribute sensitivity in people search that does a smart check for misspelled names, geo-localization and automated acronym search to add acronyms as search terms whether the user includes them in a query or not.
As with Watson Discovery, NLP and contextualization are planned to be included. An instant query prediction feature prioritizes search results based on other work immediately in progress. The context of a search is able to be further defined by referencing relevant documents that users receive from others inside and outside the organization.
Google Cloud Search
The granddaddy of all search engines pioneered the use of AI in a search engine by employing machine learning to study user queries en masse, learning the structures of the most effective ones and returning suggested query improvements to the user. It is available standalone or embedded in G Suite within the enterprise.
Google Search comes with more than 100 available connectors and integrates with collaboration, content management and data storage platforms such as Box, Microsoft’s Azure Data Lake, SharePoint and OneDrive, Amazon S3, databases such as Oracle, PostgreSQL and MySQL, and CRM systems such as Salesforce and SAP. It can even integrate with other search engines (though you may not catch Google’s sales reps promoting this too eagerly!).
Amazon announced Kendra in December 2019, and it likewise exploits NLP and ML technologies to enable more comprehensive queries, formatted not so much to surface content but to get specific answers to specific questions. In the description on the AWS web page for Kendra, an example of a question it could answer is, “How long is maternity leave?” which yields the response, “14 weeks.” It is available as a console application and can also be adopted through APIs that businesses can hook into from other applications.
Kendra can initiate domain-specific searches based on the content of a query, such as focusing a pharmaceutical question on pharmaceutical sources, IT questions on IT sources and so on, thus enriching the quality of results and filtering out less relevant ones.
Like Google Cloud and Watson, it also offers connectors to multiple data sources such as Box, OneDrive, Salesforce, Dropbox and others. And, as it runs on the global Amazon data centre footprint, performance is generally quite good wherever a user happens to be.
Lucidworks Fusion, Digital Workplace and Digital Commerce
It’s not surprising that the tech giants have all invested deeply in ML and AI to enhance their enterprise search offerings, but intriguing innovations can be found in less prominent players as well.
Earning praise from both Forrester and Gartner, Lucidworks has specialized in enterprise search, and its Fusion platform enables user-specific engine training similar to that of Einstein Search. The product is designed to be built into customer’s custom applications, and its developer-friendly tools include highly customizable UX (User Experience) components. More technically astute non-developers can also use the platform to build apps with a visual self-guided interface.
Fusion features fluid deployment of machine learning models, exploiting existing ones while accepting custom models built with Python, TensorFlow, scikit-learn and spaCy. Businesses can integrate Fusion with IBM Watson as well as conventional file systems. Lucidworks also offers custom integration as a service.
Lucidworks also offers a Digital Workplace application, which the company calls a “predictive answer engine”, made possible by merging AI-driven search with a personalization utility and collaboration features. And its Digital Commerce system offers an AI-driven personalized customer experience for eCommerce that looks to take the customer journey into account when defining the context of a query.
AlphaSense specializes in search of unstructured and fragmented data — such as email, text and social media data — and businesses can integrate the software with a number of platforms and applications via its API. The application depends heavily upon NLP and granular content classification, using AI-derived synonyms of user-provided search terms to enrich its search. Its AI architecture is scalable — with algorithms training on billions of data points — yielding refined performance with significant noise reduction.
Targeted primarily at the financial and healthcare verticals, it also has a market intelligence engine that scopes results within selected domains — much as Amazon Kendra does — and features a sentiment analysis model that can detect shifts in content tone in search results output.