org.opencms.search
Class CmsSearchManager

java.lang.Object
  extended byorg.opencms.search.CmsSearchManager
All Implemented Interfaces:
I_CmsEventListener, I_CmsScheduledJob

public class CmsSearchManager
extends java.lang.Object
implements I_CmsScheduledJob, I_CmsEventListener

Implements the general management and configuration of the search and indexing facilities in OpenCms.

Since:
6.0.0
Version:
$Revision: 1.64 $
Author:
Alexander Kandzior, Carsten Weinholz

Nested Class Summary
static class CmsSearchManager.CmsSearchForceUnlockMode
          Enumeration class for force unlock types.
 
Field Summary
static int DEFAULT_EXCERPT_LENGTH
          The default value used for generating search result exerpts (1024 chars).
static float DEFAULT_EXTRACTION_CACHE_MAX_AGE
          The default value used for keeping the extraction results in the cache (672 hours = 4 weeks).
static int DEFAULT_TIMEOUT
          The default timeout value used for generating a document for the search index (60000 msec = 1 min).
static java.lang.String JOB_PARAM_INDEXLIST
          Scheduler parameter: Update only a specified list of indexes.
static java.lang.String JOB_PARAM_WRITELOG
          Scheduler parameter: Write the output of the update to the logfile.
 
Fields inherited from interface org.opencms.main.I_CmsEventListener
EVENT_BEFORE_PUBLISH_PROJECT, EVENT_CLEAR_CACHES, EVENT_CLEAR_OFFLINE_CACHES, EVENT_CLEAR_ONLINE_CACHES, EVENT_CLEAR_PRINCIPAL_CACHES, EVENT_FLEX_CACHE_CLEAR, EVENT_FLEX_PURGE_JSP_REPOSITORY, EVENT_FULLSTATIC_EXPORT, EVENT_LOGIN_USER, EVENT_PROJECT_MODIFIED, EVENT_PROPERTY_DEFINITION_CREATED, EVENT_PROPERTY_DEFINITION_MODIFIED, EVENT_PROPERTY_MODIFIED, EVENT_PUBLISH_PROJECT, EVENT_REBUILD_SEARCHINDEXES, EVENT_RESOURCE_AND_PROPERTIES_MODIFIED, EVENT_RESOURCE_COPIED, EVENT_RESOURCE_CREATED, EVENT_RESOURCE_DELETED, EVENT_RESOURCE_LIST_MODIFIED, EVENT_RESOURCE_MODIFIED, EVENT_RESOURCE_MOVED, EVENT_RESOURCES_AND_PROPERTIES_MODIFIED, EVENT_RESOURCES_MODIFIED, EVENT_UPDATE_EXPORTS, KEY_DBCONTEXT, KEY_INDEX_NAMES, KEY_PROJECTID, KEY_PUBLISHID, KEY_PUBLISHLIST, KEY_REPORT, LISTENERS_FOR_ALL_EVENTS
 
Constructor Summary
CmsSearchManager()
          Default constructor when called as cron job.
 
Method Summary
 void addAnalyzer(CmsSearchAnalyzer analyzer)
          Adds an analyzer.
 void addDocumentTypeConfig(CmsSearchDocumentType documentType)
          Adds a document type.
 void addFieldConfiguration(CmsSearchFieldConfiguration fieldConfiguration)
          Adds a search field configuration to the search manager.
 void addSearchIndex(CmsSearchIndex searchIndex)
          Adds a search index to the configuration.
 void addSearchIndexSource(CmsSearchIndexSource searchIndexSource)
          Adds a search index source configuration.
 void cmsEvent(CmsEvent event)
          Implements the event listener of this class.
protected  void forceIndexUnlock(CmsSearchIndex index, I_CmsReport report, boolean mode)
          Proceed the unlocking of the given index depending on the setting of m_forceUnlockMode and the given mode.
protected  org.apache.lucene.analysis.Analyzer getAnalyzer(java.util.Locale locale)
          Returns an analyzer for the given language.
 java.util.Map getAnalyzers()
          Returns an unmodifiable view of the map that contains the CmsSearchAnalyzer list.
 CmsSearchAnalyzer getCmsSearchAnalyzer(java.util.Locale locale)
          Returns the search analyzer for the given locale.
 java.lang.String getDirectory()
          Returns the name of the directory below WEB-INF/ where the search indexes are stored.
protected  I_CmsDocumentFactory getDocumentFactory(CmsResource resource)
          Returns a lucene document factory for given resource.
 CmsSearchDocumentType getDocumentTypeConfig(java.lang.String name)
          Returns a document type config.
 java.util.List getDocumentTypeConfigs()
          Returns an unmodifiable view (read-only) of the DocumentTypeConfigs Map.
protected  java.util.List getDocumentTypes()
          Returns the set of names of all configured documenttypes.
 float getExtractionCacheMaxAge()
          Returns the maximum age a text extraction result is kept in the cache (in hours).
 CmsSearchFieldConfiguration getFieldConfiguration(java.lang.String name)
          Returns the search field configuration with the given name.
 java.util.List getFieldConfigurations()
          Returns the unmodifieable List of configured CmsSearchFieldConfiguration entries.
 CmsSearchManager.CmsSearchForceUnlockMode getForceunlock()
          Returns the force unlock mode during indexing.
 I_CmsTermHighlighter getHighlighter()
          Returns the highlighter.
 CmsSearchIndex getIndex(java.lang.String indexName)
          Returns the index belonging to the passed name.
 int getIndexLockMaxWaitSeconds()
          Returns the seconds to wait for an index lock during an update operation.
 java.util.List getIndexNames()
          Returns the names of all configured indexes.
 CmsSearchIndexSource getIndexSource(java.lang.String sourceName)
          Returns a search index source for a specified source name.
 int getMaxExcerptLength()
          Returns the max. excerpt length.
 java.util.List getSearchIndexes()
          Returns an unmodifiable list of all configured CmsSearchIndex instances.
 java.util.Map getSearchIndexSources()
          Returns an unmodifiable view (read-only) of the SearchIndexSources Map.
 long getTimeout()
          Returns the timeout to abandon threads indexing a resource.
protected  void initAvailableDocumentTypes()
          Initializes the available Cms resource types to be indexed.
 void initialize(CmsObject cms)
          Initializes the search manager.
 void initializeIndexes()
          Initializes all configured document types and search indexes.
protected  void initSearchIndexes()
          Initializes the configured search indexes.
 java.lang.String launch(CmsObject cms, java.util.Map parameters)
          Updates the indexes from as a scheduled job.
 void rebuildAllIndexes(I_CmsReport report)
          Rebuilds (if required creates) all configured indexes.
 void rebuildIndex(java.lang.String indexName, I_CmsReport report)
          Rebuilds (if required creates) the index with the given name.
 void rebuildIndexes(java.util.List indexNames, I_CmsReport report)
          Rebuilds (if required creates) the List of indexes with the given name.
 boolean removeSearchFieldConfiguration(CmsSearchFieldConfiguration fieldConfiguration)
          Removes this fieldconfiguration from the OpenCms configuration (if it is not used any more).
 boolean removeSearchFieldConfigurationField(CmsSearchFieldConfiguration fieldConfiguration, CmsSearchField field)
          Removes a search field from the field configuration.
 boolean removeSearchFieldMapping(CmsSearchField field, CmsSearchFieldMapping mapping)
          Removes a search field mapping from the given field.
 void removeSearchIndex(CmsSearchIndex searchIndex)
          Removes a search index from the configuration.
 void removeSearchIndexes(java.util.List indexNames)
          Removes all indexes included in the given list (which must contain the name of an index to remove).
 boolean removeSearchIndexSource(CmsSearchIndexSource indexsource)
          Removes this indexsource from the OpenCms configuration (if it is not used any more).
 void setDirectory(java.lang.String value)
          Sets the name of the directory below WEB-INF/ where the search indexes are stored.
 void setExtractionCacheMaxAge(float extractionCacheMaxAge)
          Sets the maximum age a text extraction result is kept in the cache (in hours).
 void setExtractionCacheMaxAge(java.lang.String extractionCacheMaxAge)
          Sets the maximum age a text extraction result is kept in the cache (in hours) as a String.
 void setForceunlock(java.lang.String value)
          Sets the unlock mode during indexing.
 void setHighlighter(java.lang.String highlighter)
          Sets the highlighter.
 void setIndexLockMaxWaitSeconds(int value)
          Sets the seconds to wait for an index lock during an update operation.
 void setMaxExcerptLength(int maxExcerptLength)
          Sets the max. excerpt length.
 void setMaxExcerptLength(java.lang.String maxExcerptLength)
          Sets the max. excerpt length as a String.
 void setTimeout(long value)
          Sets the timeout to abandon threads indexing a resource.
 void setTimeout(java.lang.String value)
          Sets the timeout to abandon threads indexing a resource as a String.
protected  void updateAllIndexes(CmsObject adminCms, CmsUUID publishHistoryId, I_CmsReport report)
          Incrementally updates all indexes that have their rebuild mode set to "auto" after resources have been published.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

DEFAULT_EXCERPT_LENGTH

public static final int DEFAULT_EXCERPT_LENGTH
The default value used for generating search result exerpts (1024 chars).

See Also:
Constant Field Values

DEFAULT_EXTRACTION_CACHE_MAX_AGE

public static final float DEFAULT_EXTRACTION_CACHE_MAX_AGE
The default value used for keeping the extraction results in the cache (672 hours = 4 weeks).

See Also:
Constant Field Values

DEFAULT_TIMEOUT

public static final int DEFAULT_TIMEOUT
The default timeout value used for generating a document for the search index (60000 msec = 1 min).

See Also:
Constant Field Values

JOB_PARAM_INDEXLIST

public static final java.lang.String JOB_PARAM_INDEXLIST
Scheduler parameter: Update only a specified list of indexes.

See Also:
Constant Field Values

JOB_PARAM_WRITELOG

public static final java.lang.String JOB_PARAM_WRITELOG
Scheduler parameter: Write the output of the update to the logfile.

See Also:
Constant Field Values
Constructor Detail

CmsSearchManager

public CmsSearchManager()
Default constructor when called as cron job.

Method Detail

addAnalyzer

public void addAnalyzer(CmsSearchAnalyzer analyzer)
Adds an analyzer.

Parameters:
analyzer - an analyzer

addDocumentTypeConfig

public void addDocumentTypeConfig(CmsSearchDocumentType documentType)
Adds a document type.

Parameters:
documentType - a document type

addFieldConfiguration

public void addFieldConfiguration(CmsSearchFieldConfiguration fieldConfiguration)
Adds a search field configuration to the search manager.

Parameters:
fieldConfiguration - the search field configuration to add

addSearchIndex

public void addSearchIndex(CmsSearchIndex searchIndex)
Adds a search index to the configuration.

Parameters:
searchIndex - the search index to add

addSearchIndexSource

public void addSearchIndexSource(CmsSearchIndexSource searchIndexSource)
Adds a search index source configuration.

Parameters:
searchIndexSource - a search index source configuration

cmsEvent

public void cmsEvent(CmsEvent event)
Implements the event listener of this class.

Specified by:
cmsEvent in interface I_CmsEventListener
Parameters:
event - CmsEvent that has occurred
See Also:
I_CmsEventListener.cmsEvent(org.opencms.main.CmsEvent)

getAnalyzers

public java.util.Map getAnalyzers()
Returns an unmodifiable view of the map that contains the CmsSearchAnalyzer list.

The keys in the map are Locale objects, and the values are CmsSearchAnalyzer objects.

Returns:
an unmodifiable view of the Analyzers Map

getCmsSearchAnalyzer

public CmsSearchAnalyzer getCmsSearchAnalyzer(java.util.Locale locale)
Returns the search analyzer for the given locale.

Parameters:
locale - the locale to get the analyzer for
Returns:
the search analyzer for the given locale

getDirectory

public java.lang.String getDirectory()
Returns the name of the directory below WEB-INF/ where the search indexes are stored.

Returns:
the name of the directory below WEB-INF/ where the search indexes are stored

getDocumentTypeConfig

public CmsSearchDocumentType getDocumentTypeConfig(java.lang.String name)
Returns a document type config.

Parameters:
name - the name of the document type config
Returns:
the document type config.

getDocumentTypeConfigs

public java.util.List getDocumentTypeConfigs()
Returns an unmodifiable view (read-only) of the DocumentTypeConfigs Map.

Returns:
an unmodifiable view (read-only) of the DocumentTypeConfigs Map

getExtractionCacheMaxAge

public float getExtractionCacheMaxAge()
Returns the maximum age a text extraction result is kept in the cache (in hours).

Returns:
the maximum age a text extraction result is kept in the cache (in hours)

getFieldConfiguration

public CmsSearchFieldConfiguration getFieldConfiguration(java.lang.String name)
Returns the search field configuration with the given name.

In case no configuration is available with the given name, null is returned.

Parameters:
name - the name to get the search field configuration for
Returns:
the search field configuration with the given name

getFieldConfigurations

public java.util.List getFieldConfigurations()
Returns the unmodifieable List of configured CmsSearchFieldConfiguration entries.

Returns:
the unmodifieable List of configured CmsSearchFieldConfiguration entries

getForceunlock

public CmsSearchManager.CmsSearchForceUnlockMode getForceunlock()
Returns the force unlock mode during indexing.

Returns:
the force unlock mode during indexing

getHighlighter

public I_CmsTermHighlighter getHighlighter()
Returns the highlighter.

Returns:
the highlighter

getIndex

public CmsSearchIndex getIndex(java.lang.String indexName)
Returns the index belonging to the passed name.

The index must exist already.

Parameters:
indexName - then name of the index
Returns:
an object representing the desired index

getIndexLockMaxWaitSeconds

public int getIndexLockMaxWaitSeconds()
Returns the seconds to wait for an index lock during an update operation.

Returns:
the seconds to wait for an index lock during an update operation

getIndexNames

public java.util.List getIndexNames()
Returns the names of all configured indexes.

Returns:
list of names

getIndexSource

public CmsSearchIndexSource getIndexSource(java.lang.String sourceName)
Returns a search index source for a specified source name.

Parameters:
sourceName - the name of the index source
Returns:
a search index source

getMaxExcerptLength

public int getMaxExcerptLength()
Returns the max. excerpt length.

Returns:
the max excerpt length

getSearchIndexes

public java.util.List getSearchIndexes()
Returns an unmodifiable list of all configured CmsSearchIndex instances.

Returns:
an unmodifiable list of all configured CmsSearchIndex instances

getSearchIndexSources

public java.util.Map getSearchIndexSources()
Returns an unmodifiable view (read-only) of the SearchIndexSources Map.

Returns:
an unmodifiable view (read-only) of the SearchIndexSources Map

getTimeout

public long getTimeout()
Returns the timeout to abandon threads indexing a resource.

Returns:
the timeout to abandon threads indexing a resource

initialize

public void initialize(CmsObject cms)
                throws CmsRoleViolationException
Initializes the search manager.

Parameters:
cms - the cms object
Throws:
CmsRoleViolationException - in case the given opencms object does not have CmsRole.WORKPLACE_MANAGER permissions

initializeIndexes

public void initializeIndexes()
Initializes all configured document types and search indexes.

This methods needs to be called if after a change in the index configuration has been made.


launch

public java.lang.String launch(CmsObject cms,
                               java.util.Map parameters)
                        throws java.lang.Exception
Updates the indexes from as a scheduled job.

Specified by:
launch in interface I_CmsScheduledJob
Parameters:
cms - the OpenCms user context to use when reading resources from the VFS
parameters - the parameters for the scheduled job
Returns:
the String to write in the scheduler log
Throws:
java.lang.Exception - if something goes wrong
See Also:
I_CmsScheduledJob.launch(org.opencms.file.CmsObject, java.util.Map)

rebuildAllIndexes

public void rebuildAllIndexes(I_CmsReport report)
                       throws CmsException
Rebuilds (if required creates) all configured indexes.

Parameters:
report - the report object to write messages (or null)
Throws:
CmsException - if something goes wrong

rebuildIndex

public void rebuildIndex(java.lang.String indexName,
                         I_CmsReport report)
                  throws CmsException
Rebuilds (if required creates) the index with the given name.

Parameters:
indexName - the name of the index to rebuild
report - the report object to write messages (or null)
Throws:
CmsException - if something goes wrong

rebuildIndexes

public void rebuildIndexes(java.util.List indexNames,
                           I_CmsReport report)
                    throws CmsException
Rebuilds (if required creates) the List of indexes with the given name.

Parameters:
indexNames - the names (String) of the index to rebuild
report - the report object to write messages (or null)
Throws:
CmsException - if something goes wrong

removeSearchFieldConfiguration

public boolean removeSearchFieldConfiguration(CmsSearchFieldConfiguration fieldConfiguration)
                                       throws CmsIllegalStateException
Removes this fieldconfiguration from the OpenCms configuration (if it is not used any more).

Parameters:
fieldConfiguration - the fieldconfiguration to remove from the configuration
Returns:
true if remove was successful, false if preconditions for removal are ok but the given field configuration was unknown to the manager.
Throws:
CmsIllegalStateException - if the given field configuration is still used by at least one CmsSearchIndex.

removeSearchFieldConfigurationField

public boolean removeSearchFieldConfigurationField(CmsSearchFieldConfiguration fieldConfiguration,
                                                   CmsSearchField field)
                                            throws CmsIllegalStateException
Removes a search field from the field configuration.

Parameters:
fieldConfiguration - the field configuration
field - field to remove from the field configuration
Returns:
true if remove was successful, false if preconditions for removal are ok but the given field was unknown.
Throws:
CmsIllegalStateException - if the given field is the last field inside the given field configuration.

removeSearchFieldMapping

public boolean removeSearchFieldMapping(CmsSearchField field,
                                        CmsSearchFieldMapping mapping)
                                 throws CmsIllegalStateException
Removes a search field mapping from the given field.

Parameters:
field - the field
mapping - mapping to remove from the field
Returns:
true if remove was successful, false if preconditions for removal are ok but the given mapping was unknown.
Throws:
CmsIllegalStateException - if the given mapping is the last mapping inside the given field.

removeSearchIndex

public void removeSearchIndex(CmsSearchIndex searchIndex)
Removes a search index from the configuration.

Parameters:
searchIndex - the search index to remove

removeSearchIndexes

public void removeSearchIndexes(java.util.List indexNames)
Removes all indexes included in the given list (which must contain the name of an index to remove).

Parameters:
indexNames - the names of the index to remove

removeSearchIndexSource

public boolean removeSearchIndexSource(CmsSearchIndexSource indexsource)
                                throws CmsIllegalStateException
Removes this indexsource from the OpenCms configuration (if it is not used any more).

Parameters:
indexsource - the indexsource to remove from the configuration
Returns:
true if remove was successful, false if preconditions for removal are ok but the given searchindex was unknown to the manager.
Throws:
CmsIllegalStateException - if the given indexsource is still used by at least one CmsSearchIndex.

setDirectory

public void setDirectory(java.lang.String value)
Sets the name of the directory below WEB-INF/ where the search indexes are stored.

Parameters:
value - the name of the directory below WEB-INF/ where the search indexes are stored

setExtractionCacheMaxAge

public void setExtractionCacheMaxAge(float extractionCacheMaxAge)
Sets the maximum age a text extraction result is kept in the cache (in hours).

Parameters:
extractionCacheMaxAge - the maximum age for a text extraction result to set

setExtractionCacheMaxAge

public void setExtractionCacheMaxAge(java.lang.String extractionCacheMaxAge)
Sets the maximum age a text extraction result is kept in the cache (in hours) as a String.

Parameters:
extractionCacheMaxAge - the maximum age for a text extraction result to set

setForceunlock

public void setForceunlock(java.lang.String value)
Sets the unlock mode during indexing.

Parameters:
value - the value

setHighlighter

public void setHighlighter(java.lang.String highlighter)
Sets the highlighter.

A highlighter is a class implementing org.opencms.search.documents.I_TermHighlighter.

Parameters:
highlighter - the package/class name of the highlighter

setIndexLockMaxWaitSeconds

public void setIndexLockMaxWaitSeconds(int value)
Sets the seconds to wait for an index lock during an update operation.

Parameters:
value - the seconds to wait for an index lock during an update operation

setMaxExcerptLength

public void setMaxExcerptLength(int maxExcerptLength)
Sets the max. excerpt length.

Parameters:
maxExcerptLength - the max. excerpt length to set

setMaxExcerptLength

public void setMaxExcerptLength(java.lang.String maxExcerptLength)
Sets the max. excerpt length as a String.

Parameters:
maxExcerptLength - the max. excerpt length to set

setTimeout

public void setTimeout(long value)
Sets the timeout to abandon threads indexing a resource.

Parameters:
value - the timeout in milliseconds

setTimeout

public void setTimeout(java.lang.String value)
Sets the timeout to abandon threads indexing a resource as a String.

Parameters:
value - the timeout in milliseconds

forceIndexUnlock

protected void forceIndexUnlock(CmsSearchIndex index,
                                I_CmsReport report,
                                boolean mode)
                         throws CmsIndexException
Proceed the unlocking of the given index depending on the setting of m_forceUnlockMode and the given mode.

Parameters:
index - the index to check the lock for
report - the report to write error messages on
mode - the mode of the index process if true the index is updated otherwise it is rebuild completely
Throws:
CmsIndexException - if unlocking of the index is impossible for some reasons

getAnalyzer

protected org.apache.lucene.analysis.Analyzer getAnalyzer(java.util.Locale locale)
                                                   throws CmsIndexException
Returns an analyzer for the given language.

The analyzer is selected according to the analyzer configuration.

Parameters:
locale - the locale to get the analyzer for
Returns:
the appropriate lucene analyzer
Throws:
CmsIndexException - if something goes wrong

getDocumentFactory

protected I_CmsDocumentFactory getDocumentFactory(CmsResource resource)
Returns a lucene document factory for given resource.

The type of the document factory is selected by the type of the resource and the MIME type of the resource content, according to the configuration in opencms-search.xml.

Parameters:
resource - a cms resource
Returns:
a lucene document factory or null

getDocumentTypes

protected java.util.List getDocumentTypes()
Returns the set of names of all configured documenttypes.

Returns:
the set of names of all configured documenttypes

initAvailableDocumentTypes

protected void initAvailableDocumentTypes()
Initializes the available Cms resource types to be indexed.

A map stores document factories keyed by a string representing a colon separated list of Cms resource types and/or mimetypes.

The keys of this map are used to trigger a document factory to convert a Cms resource into a Lucene index document.

A document factory is a class implementing the interface I_CmsDocumentFactory.


initSearchIndexes

protected void initSearchIndexes()
Initializes the configured search indexes.

This initializes also the list of Cms resources types to be indexed by an index source.


updateAllIndexes

protected void updateAllIndexes(CmsObject adminCms,
                                CmsUUID publishHistoryId,
                                I_CmsReport report)
Incrementally updates all indexes that have their rebuild mode set to "auto" after resources have been published.

Parameters:
adminCms - an OpenCms user context with Admin permissions
publishHistoryId - the history ID of the published project
report - the report to write the output to