Main Page   Class Hierarchy   Alphabetical List   Compound List   File List   Compound Members   File Members   Search  

RuleBasedBreakIterator Class Reference

A subclass of BreakIterator whose behavior is specified using a list of rules. More...

#include <rbbi.h>

Inheritance diagram for RuleBasedBreakIterator::

BreakIterator UObject UMemory DictionaryBasedBreakIterator List of all members.

Public Methods

 RuleBasedBreakIterator ()
 Default constructor. More...

 RuleBasedBreakIterator (const RuleBasedBreakIterator &that)
 Copy constructor. More...

 RuleBasedBreakIterator (const UnicodeString &rules, UParseError &parseError, UErrorCode &status)
 Construct a RuleBasedBreakIterator from a set of rules supplied as a string. More...

 RuleBasedBreakIterator (UDataMemory *image, UErrorCode &status)
 This constructor uses the udata interface to create a BreakIterator whose internal tables live in a memory-mapped file. More...

virtual ~RuleBasedBreakIterator ()
 Destructor. More...

RuleBasedBreakIterator & operator= (const RuleBasedBreakIterator &that)
 Assignment operator. More...

virtual UBool operator== (const BreakIterator &that) const
 Equality operator. More...

UBool operator!= (const BreakIterator &that) const
 Not-equal operator. More...

virtual BreakIteratorclone () const
 Returns a newly-constructed RuleBasedBreakIterator with the same behavior, and iterating over the same text, as this one. More...

virtual int32_t hashCode (void) const
 Compute a hash code for this BreakIterator. More...

virtual const UnicodeStringgetRules (void) const
 Returns the description used to create this iterator. More...

virtual const CharacterIteratorgetText (void) const
 Return a CharacterIterator over the text being analyzed. More...

virtual void adoptText (CharacterIterator *newText)
 Set the iterator to analyze a new piece of text. More...

virtual void setText (const UnicodeString &newText)
 Set the iterator to analyze a new piece of text. More...

virtual int32_t first (void)
 Sets the current iteration position to the beginning of the text. More...

virtual int32_t last (void)
 Sets the current iteration position to the end of the text. More...

virtual int32_t next (int32_t n)
 Advances the iterator either forward or backward the specified number of steps. More...

virtual int32_t next (void)
 Advances the iterator to the next boundary position. More...

virtual int32_t previous (void)
 Moves the iterator backwards, to the last boundary preceding this one. More...

virtual int32_t following (int32_t offset)
 Sets the iterator to refer to the first boundary position following the specified position. More...

virtual int32_t preceding (int32_t offset)
 Sets the iterator to refer to the last boundary position before the specified position. More...

virtual UBool isBoundary (int32_t offset)
 Returns true if the specfied position is a boundary position. More...

virtual int32_t current (void) const
 Returns the current iteration position. More...

virtual int32_t getRuleStatus () const
 Return the status tag from the break rule that determined the most recently returned break position. More...

virtual UClassID getDynamicClassID (void) const
 Returns a unique class ID POLYMORPHICALLY. More...

virtual BreakIteratorcreateBufferClone (void *stackBuffer, int32_t &BufferSize, UErrorCode &status)
 Thread safe client-buffer-based cloning operation Do NOT call delete on a safeclone, since 'new' is not used to create it. More...

virtual const uint8_t * getBinaryRules (uint32_t &length)
 Return the binary form of compiled break rules, which can then be used to create a new break iterator at some time in the future. More...


Static Public Methods

UClassID getStaticClassID (void)
 Returns the class ID for this class. More...


Protected Methods

 RuleBasedBreakIterator (RBBIDataHeader *data, UErrorCode &status)
 Constructor from a flattened set of RBBI data in malloced memory. More...

virtual int32_t handleNext (void)
 This method is the actual implementation of the next() method. More...

virtual int32_t handlePrevious (void)
 This method backs the iterator back up to a "safe position" in the text. More...

virtual void reset (void)
 Dumps caches and performs other actions associated with a complete change in text or iteration position. More...

virtual UBool isDictionaryChar (UChar32)
 Return true if the category lookup for this char indicates that it is in the set of dictionary lookup chars. More...

void init ()
 Common initialization function, used by constructors and bufferClone. More...


Protected Attributes

CharacterIteratorfText
 The character iterator through which this BreakIterator accesses the text. More...

RBBIDataWrapper * fData
 The rule data for this BreakIterator instance. More...

int32_t fLastBreakTag
 Rule {tag} value for the most recent match. More...

UBool fLastBreakTagValid
 Rule tag value valid flag. More...

uint32_t fDictionaryCharCount
 Counter for the number of characters encountered with the "dictionary" flag set. More...


Static Protected Attributes

UBool fTrace
 Debugging flag. More...


Private Methods

int32_t handlePrevious (const RBBIStateTable *statetable)
 This method backs the iterator back up to a "safe position" in the text. More...

int32_t handleNext (const RBBIStateTable *statetable)
 This method is the actual implementation of the next() method. More...


Friends

class RBBIRuleBuilder
class BreakIterator

Detailed Description

A subclass of BreakIterator whose behavior is specified using a list of rules.

Instances of this class are most commonly created by the factory methods of BreakIterator::createWordInstance(), BreakIterator::createLineInstance(), etc., and then used via the abstract API in class BreakIterator

See the ICU User Guide for information on Break Iterator Rules.

This class is not intended to be subclassed. (Class DictionaryBasedBreakIterator is a subclass, but that relationship is effectively internal to the ICU implementation. The subclassing interface to RulesBasedBreakIterator is not part of the ICU API, and may not remain stable.

Definition at line 51 of file rbbi.h.


Constructor & Destructor Documentation

RuleBasedBreakIterator::RuleBasedBreakIterator RBBIDataHeader   data,
UErrorCode   status
[protected]
 

Constructor from a flattened set of RBBI data in malloced memory.

RulesBasedBreakIterators built from a custom set of rules are created via this constructor; the rules are compiled into memory, then the break iterator is constructed here.

The break iterator adopts the memory, and will free it when done.

Internal:
For internal use only.

RuleBasedBreakIterator::RuleBasedBreakIterator  
 

Default constructor.

Creates an empty shell of an iterator, with no rules or text to iterate over. Object can subsequently be assigned to.

Stable:
ICU 2.2

RuleBasedBreakIterator::RuleBasedBreakIterator const RuleBasedBreakIterator &    that
 

Copy constructor.

Will produce a break iterator with the same behavior, and which iterates over the same text, as the one passed in.

Parameters:
that  The RuleBasedBreakIterator passed to be copied
Stable:
ICU 2.0

RuleBasedBreakIterator::RuleBasedBreakIterator const UnicodeString   rules,
UParseError   parseError,
UErrorCode   status
 

Construct a RuleBasedBreakIterator from a set of rules supplied as a string.

Parameters:
rules  The break rules to be used.
parseError  In the event of a syntax error in the rules, provides the location within the rules of the problem.
status  Information on any errors encountered.
Stable:
ICU 2.2

RuleBasedBreakIterator::RuleBasedBreakIterator UDataMemory   image,
UErrorCode   status
 

This constructor uses the udata interface to create a BreakIterator whose internal tables live in a memory-mapped file.

"image" is an ICU UDataMemory handle for the pre-compiled break iterator tables.

Parameters:
image  handle to the memory image for the break iterator data.
status  Information on any errors encountered.
See also:
udata_open , getBinaryRules
Draft:
This API has been introduced in ICU 2.8. It is still in draft state and may be modified in a future release.

virtual RuleBasedBreakIterator::~RuleBasedBreakIterator   [virtual]
 

Destructor.

Stable:
ICU 2.0


Member Function Documentation

virtual void RuleBasedBreakIterator::adoptText CharacterIterator   newText [virtual]
 

Set the iterator to analyze a new piece of text.

This function resets the current iteration position to the beginning of the text.

Parameters:
newText  An iterator over the text to analyze. The BreakIterator takes ownership of the character iterator. The caller MUST NOT delete it!
Stable:
ICU 2.0

Reimplemented from BreakIterator.

virtual BreakIterator* RuleBasedBreakIterator::clone void    const [virtual]
 

Returns a newly-constructed RuleBasedBreakIterator with the same behavior, and iterating over the same text, as this one.

Differs from the copy constructor in that it is polymorphic, and will correctly clone (copy) a derived class. clone() is thread safe. Multiple threads may simultaeneously clone the same source break iterator.

Stable:
ICU 2.0

Reimplemented from BreakIterator.

Reimplemented in DictionaryBasedBreakIterator.

virtual BreakIterator* RuleBasedBreakIterator::createBufferClone void *    stackBuffer,
int32_t &    BufferSize,
UErrorCode   status
[virtual]
 

Thread safe client-buffer-based cloning operation Do NOT call delete on a safeclone, since 'new' is not used to create it.

Parameters:
stackBuffer  user allocated space for the new clone. If NULL new memory will be allocated. If buffer is not large enough, new memory will be allocated.
BufferSize  reference to size of allocated space. If BufferSize == 0, a sufficient size for use in cloning will be returned ('pre-flighting') If BufferSize is not enough for a stack-based safe clone, new memory will be allocated.
status  to indicate whether the operation went on smoothly or there were errors An informational status value, U_SAFECLONE_ALLOCATED_ERROR, is used if any allocations were necessary.
Returns:
pointer to the new clone
Stable:
ICU 2.0

Reimplemented from BreakIterator.

Reimplemented in DictionaryBasedBreakIterator.

virtual int32_t RuleBasedBreakIterator::current void    const [virtual]
 

Returns the current iteration position.

Returns:
The current iteration position.
Stable:
ICU 2.0

Reimplemented from BreakIterator.

virtual int32_t RuleBasedBreakIterator::first void    [virtual]
 

Sets the current iteration position to the beginning of the text.

(i.e., the CharacterIterator's starting offset).

Returns:
The offset of the beginning of the text.
Stable:
ICU 2.0

Reimplemented from BreakIterator.

virtual int32_t RuleBasedBreakIterator::following int32_t    offset [virtual]
 

Sets the iterator to refer to the first boundary position following the specified position.

Parameters:
offset  The position from which to begin searching for a break position.
Returns:
The position of the first break after the current position.
Stable:
ICU 2.0

Reimplemented from BreakIterator.

Reimplemented in DictionaryBasedBreakIterator.

virtual const uint8_t* RuleBasedBreakIterator::getBinaryRules uint32_t &    length [virtual]
 

Return the binary form of compiled break rules, which can then be used to create a new break iterator at some time in the future.

Creating a break iterator from pre-compiled rules is much faster than building one from the source form of the break rules.

The binary data can only be used with the same version of ICU and on the same platform type (processor endian-ness)

Parameters:
length  Returns the length of the binary data. (Out paramter.)
Returns:
A pointer to the binary (compiled) rule data. The storage belongs to the RulesBasedBreakIterator object, not the caller, and must not be modified or deleted.
Internal:
For internal use only.

virtual UClassID RuleBasedBreakIterator::getDynamicClassID void    const [virtual]
 

Returns a unique class ID POLYMORPHICALLY.

Pure virtual override. This method is to implement a simple version of RTTI, since not all C++ compilers support genuine RTTI. Polymorphic operator==() and clone() methods call this method.

Returns:
The class ID for this object. All objects of a given class have the same class ID. Objects of other classes have different class IDs.
Stable:
ICU 2.0

Reimplemented from BreakIterator.

Reimplemented in DictionaryBasedBreakIterator.

virtual int32_t RuleBasedBreakIterator::getRuleStatus   const [virtual]
 

Return the status tag from the break rule that determined the most recently returned break position.

The values appear in the rule source within brackets, {123}, for example. For rules that do not specify a status, a default value of 0 is returned.

Of the standard types of ICU break iterators, only the word break iterator provides status values. The values are defined in enum UWordBreak, and allow distinguishing between words that contain alphabetic letters, "words" that appear to be numbers, punctuation and spaces, words containing ideographic characters, and more. Call getRuleStatus after obtaining a boundary position from next(), previous(), or any other break iterator functions that returns a boundary position.

Returns:
the status from the break rule that determined the most recently returned break position.
See also:
UWordBreak
Stable:
ICU 2.2

virtual const UnicodeString& RuleBasedBreakIterator::getRules void    const [virtual]
 

Returns the description used to create this iterator.

Returns:
the description used to create this iterator
Stable:
ICU 2.0

UClassID RuleBasedBreakIterator::getStaticClassID void    [static]
 

Returns the class ID for this class.

This is useful only for comparing to a return value from getDynamicClassID(). For example:

Base* polymorphic_pointer = createPolymorphicObject(); if (polymorphic_pointer->getDynamicClassID() == Derived::getStaticClassID()) ...

Returns:
The class ID for all objects of this class.
Stable:
ICU 2.0

Reimplemented in DictionaryBasedBreakIterator.

virtual const CharacterIterator& RuleBasedBreakIterator::getText void    const [virtual]
 

Return a CharacterIterator over the text being analyzed.

This version of this method returns the actual CharacterIterator we're using internally. Changing the state of this iterator can have undefined consequences. If you need to change it, clone it first.

Returns:
An iterator over the text being analyzed.
Stable:
ICU 2.0

Reimplemented from BreakIterator.

int32_t RuleBasedBreakIterator::handleNext const RBBIStateTable *    statetable [private]
 

This method is the actual implementation of the next() method.

All iteration vectors through here. This method initializes the state machine to state 1 and advances through the text character by character until we reach the end of the text or the state machine transitions to state 0. We update our return value every time the state machine passes through a possible end state.

Parameters:
statetable  state table used of moving forwards
Internal:
For internal use only.

virtual int32_t RuleBasedBreakIterator::handleNext void    [protected, virtual]
 

This method is the actual implementation of the next() method.

All iteration vectors through here. This method initializes the state machine to state 1 and advances through the text character by character until we reach the end of the text or the state machine transitions to state 0. We update our return value every time the state machine passes through a possible end state.

Internal:
For internal use only.

Reimplemented in DictionaryBasedBreakIterator.

int32_t RuleBasedBreakIterator::handlePrevious const RBBIStateTable *    statetable [private]
 

This method backs the iterator back up to a "safe position" in the text.

This is a position that we know, without any context, must be a break position. The various calling methods then iterate forward from this safe position to the appropriate position to return. (For more information, see the description of buildBackwardsStateTable() in RuleBasedBreakIterator.Builder.)

Parameters:
statetable  state table used of moving backwards
Internal:
For internal use only.

virtual int32_t RuleBasedBreakIterator::handlePrevious void    [protected, virtual]
 

This method backs the iterator back up to a "safe position" in the text.

This is a position that we know, without any context, must be a break position. The various calling methods then iterate forward from this safe position to the appropriate position to return. (For more information, see the description of buildBackwardsStateTable() in RuleBasedBreakIterator.Builder.)

Internal:
For internal use only.

virtual int32_t RuleBasedBreakIterator::hashCode void    const [virtual]
 

Compute a hash code for this BreakIterator.

Returns:
A hash code
Stable:
ICU 2.0

void RuleBasedBreakIterator::init   [protected]
 

Common initialization function, used by constructors and bufferClone.

(Also used by DictionaryBasedBreakIterator::createBufferClone().)

Internal:
For internal use only.

Reimplemented in DictionaryBasedBreakIterator.

virtual UBool RuleBasedBreakIterator::isBoundary int32_t    offset [virtual]
 

Returns true if the specfied position is a boundary position.

As a side effect, leaves the iterator pointing to the first boundary position at or after "offset".

Parameters:
offset  the offset to check.
Returns:
True if "offset" is a boundary position.
Stable:
ICU 2.0

Reimplemented from BreakIterator.

virtual UBool RuleBasedBreakIterator::isDictionaryChar UChar32    [protected, virtual]
 

Return true if the category lookup for this char indicates that it is in the set of dictionary lookup chars.

This function is intended for use by dictionary based break iterators.

Returns:
true if the category lookup for this char indicates that it is in the set of dictionary lookup chars.
Internal:
For internal use only.

virtual int32_t RuleBasedBreakIterator::last void    [virtual]
 

Sets the current iteration position to the end of the text.

(i.e., the CharacterIterator's ending offset).

Returns:
The text's past-the-end offset.
Stable:
ICU 2.0

Reimplemented from BreakIterator.

virtual int32_t RuleBasedBreakIterator::next void    [virtual]
 

Advances the iterator to the next boundary position.

Returns:
The position of the first boundary after this one.
Stable:
ICU 2.0

Reimplemented from BreakIterator.

virtual int32_t RuleBasedBreakIterator::next int32_t    n [virtual]
 

Advances the iterator either forward or backward the specified number of steps.

Negative values move backward, and positive values move forward. This is equivalent to repeatedly calling next() or previous().

Parameters:
n  The number of steps to move. The sign indicates the direction (negative is backwards, and positive is forwards).
Returns:
The character offset of the boundary position n boundaries away from the current one.
Stable:
ICU 2.0

Reimplemented from BreakIterator.

UBool RuleBasedBreakIterator::operator!= const BreakIterator   that const [inline]
 

Not-equal operator.

If operator== returns TRUE, this returns FALSE, and vice versa.

Parameters:
that  The BreakIterator to be compared for inequality
Returns:
TRUE if both BreakIterators are not same.
Stable:
ICU 2.0

Reimplemented from BreakIterator.

Definition at line 505 of file rbbi.h.

RuleBasedBreakIterator& RuleBasedBreakIterator::operator= const RuleBasedBreakIterator &    that
 

Assignment operator.

Sets this iterator to have the same behavior, and iterate over the same text, as the one passed in.

Parameters:
that  The RuleBasedBreakItertor passed in
Returns:
the newly created RuleBasedBreakIterator
Stable:
ICU 2.0

virtual UBool RuleBasedBreakIterator::operator== const BreakIterator   that const [virtual]
 

Equality operator.

Returns TRUE if both BreakIterators are of the same class, have the same behavior, and iterate over the same text.

Parameters:
that  The BreakIterator to be compared for equality @Return TRUE if both BreakIterators are of the same class, have the same behavior, and iterate over the same text.
Stable:
ICU 2.0

Reimplemented from BreakIterator.

Referenced by operator!=().

virtual int32_t RuleBasedBreakIterator::preceding int32_t    offset [virtual]
 

Sets the iterator to refer to the last boundary position before the specified position.

Parameters:
offset  The position to begin searching for a break from.
Returns:
The position of the last boundary before the starting position.
Stable:
ICU 2.0

Reimplemented from BreakIterator.

Reimplemented in DictionaryBasedBreakIterator.

virtual int32_t RuleBasedBreakIterator::previous void    [virtual]
 

Moves the iterator backwards, to the last boundary preceding this one.

Returns:
The position of the last boundary position preceding this one.
Stable:
ICU 2.0

Reimplemented from BreakIterator.

Reimplemented in DictionaryBasedBreakIterator.

virtual void RuleBasedBreakIterator::reset void    [protected, virtual]
 

Dumps caches and performs other actions associated with a complete change in text or iteration position.

This function is a no-op in RuleBasedBreakIterator, but subclasses can and do override it.

Internal:
For internal use only.

Reimplemented in DictionaryBasedBreakIterator.

virtual void RuleBasedBreakIterator::setText const UnicodeString   newText [virtual]
 

Set the iterator to analyze a new piece of text.

This function resets the current iteration position to the beginning of the text.

Parameters:
newText  The text to analyze.
Stable:
ICU 2.0

Reimplemented from BreakIterator.


Friends And Related Function Documentation

friend class BreakIterator [friend]
 

Internal:
For internal use only.

Reimplemented in DictionaryBasedBreakIterator.

Definition at line 113 of file rbbi.h.

friend class RBBIRuleBuilder [friend]
 

Definition at line 112 of file rbbi.h.


Member Data Documentation

RBBIDataWrapper* RuleBasedBreakIterator::fData [protected]
 

The rule data for this BreakIterator instance.

Internal:
For internal use only.

Definition at line 64 of file rbbi.h.

uint32_t RuleBasedBreakIterator::fDictionaryCharCount [protected]
 

Counter for the number of characters encountered with the "dictionary" flag set.

Normal RBBI iterators don't use it, although the code for updating it is live. Dictionary Based break iterators (a subclass of us) access this field directly.

Internal:
For internal use only.

Definition at line 86 of file rbbi.h.

int32_t RuleBasedBreakIterator::fLastBreakTag [protected]
 

Rule {tag} value for the most recent match.

Internal:
For internal use only.

Definition at line 69 of file rbbi.h.

UBool RuleBasedBreakIterator::fLastBreakTagValid [protected]
 

Rule tag value valid flag.

Some iterator operations don't intrinsically set the correct tag value. This flag lets us lazily compute the value if we are ever asked for it.

Internal:
For internal use only.

Definition at line 77 of file rbbi.h.

CharacterIterator* RuleBasedBreakIterator::fText [protected]
 

The character iterator through which this BreakIterator accesses the text.

Internal:
For internal use only.

Definition at line 58 of file rbbi.h.

UBool RuleBasedBreakIterator::fTrace [static, protected]
 

Debugging flag.

Trace operation of state machine when true.

Internal:
For internal use only.

Definition at line 92 of file rbbi.h.


The documentation for this class was generated from the following file:
Generated on Mon Nov 24 14:36:47 2003 for ICU 2.8 by doxygen1.2.11.1 written by Dimitri van Heesch, © 1997-2001