1.

A string to be processed ($_ if the string is omitted or "undef")

2.

A string specifying a pattern to be matched as a prefix (which is to be skipped). If omitted, optional whitespace is skipped.

On success in a list context, an array of 3 elements is returned. The elements are:

[0]

the extracted variable, or variablish expression

[1]

the remainder of the input text,

[2]

the prefix substring (if any),

On failure, all of these values (except the remaining text) are "undef".

In a scalar context, "extract_variable" returns just the complete substring that matched a variablish expression. "undef" is returned on failure. In addition, the original input text has the returned substring (and any prefix) removed from it.

In a void context, the input text just has the matched substring (and any specified prefix) removed.

1.

A string to be processed ($_ if the string is omitted or "undef")

2.

A string specifying a pattern (i.e. regex) to be matched as the opening tag. If the pattern string is omitted (or "undef") then a pattern that matches any standard XML tag is used.

3.

A string specifying a pattern to be matched at the closing tag. If the pattern string is omitted (or "undef") then the closing tag is constructed by inserting a "/" after any leading bracket characters in the actual opening tag that was matched (not the pattern that matched the tag). For example, if the opening tag pattern is specified as '{{\w+}}' and actually matched the opening tag "{{DATA}}", then the constructed closing tag would be "{{/DATA}}".

4.

A string specifying a pattern to be matched as a prefix (which is to be skipped). If omitted, optional whitespace is skipped.

5.

A hash reference containing various parsing options (see below)

The various options that can be specified are:

"reject => $listref"

The list reference contains one or more strings specifying patterns that must not appear within the tagged text.

For example, to extract an HTML link (which should not contain nested links) use:

        extract_tagged($text, '<A>', '</A>', undef, {reject => ['<A>']} );
    

"ignore => $listref"

The list reference contains one or more strings specifying patterns that are not to be treated as nested tags within the tagged text (even if they would match the start tag pattern).

For example, to extract an arbitrary XML tag, but ignore "empty" elements:

        extract_tagged($text, undef, undef, undef, {ignore => ['<[^>]*/>']} );
    

(also see "gen_delimited_pat" below).

"fail => $str"

The "fail" option indicates the action to be taken if a matching end tag is not encountered (i.e. before the end of the string or some "reject" pattern matches). By default, a failure to match a closing tag causes "extract_tagged" to immediately fail.

However, if the string value associated with <reject> is "MAX", then "extract_tagged" returns the complete text up to the point of failure. If the string is "PARA", "extract_tagged" returns only the first paragraph after the tag (up to the first line that is either empty or contains only whitespace characters). If the string is "", the default behaviour (i.e. failure) is reinstated.

For example, suppose the start tag "/para" introduces a paragraph, which then continues until the next "/endpara" tag or until another "/para" tag is encountered:

        $text = "/para line 1\n\nline 3\n/para line 4";
        extract_tagged($text, '/para', '/endpara', undef,
                                {reject => '/para', fail => MAX );
        # EXTRACTED: "/para line 1\n\nline 3\n"
    

Suppose instead, that if no matching "/endpara" tag is found, the "/para" tag refers only to the immediately following paragraph:

        $text = "/para line 1\n\nline 3\n/para line 4";
        extract_tagged($text, '/para', '/endpara', undef,
                        {reject => '/para', fail => MAX );
        # EXTRACTED: "/para line 1\n"
    

Note that the specified "fail" behaviour applies to nested tags as well.

On success in a list context, an array of 6 elements is returned. The elements are:

[0]

the extracted tagged substring (including the outermost tags),

[1]

the remainder of the input text,

[2]

the prefix substring (if any),

[3]

the opening tag

[4]

the text between the opening and closing tags

[5]

the closing tag (or "" if no closing tag was found)

On failure, all of these values (except the remaining text) are "undef".

In a scalar context, "extract_tagged" returns just the complete substring that matched a tagged text (including the start and end tags). "undef" is returned on failure. In addition, the original input text has the returned substring (and any prefix) removed from it.

In a void context, the input text just has the matched substring (and any specified prefix) removed.

• do not have to reparse tag specification or parsing options every time they are called (whereas "extract_tagged" has to effectively rebuild its tag parser on every call);
• make use of the new qr// construct to pre-compile the regexes they use (whereas "extract_tagged" uses standard string variable interpolation to create tag-matching patterns).

The subroutine takes up to four optional arguments (the same set as "extract_tagged" except for the string to be processed). It returns a reference to a subroutine which in turn takes a single argument (the text to be extracted from).

In other words, the implementation of "extract_tagged" is exactly equivalent to:

        sub extract_tagged
        {
                my $text = shift;
                $extractor = gen_extract_tagged(@_);
                return $extractor->($text);
        }

(although "extract_tagged" is not currently implemented that way).

Using "gen_extract_tagged" to create extraction functions for specific tags is a good idea if those functions are going to be called more than once, since their performance is typically twice as good as the more general-purpose "extract_tagged".

[0]

the extracted quotelike substring (including trailing modifiers),

[1]

the remainder of the input text,

[2]

the prefix substring (if any),

[3]

the name of the quotelike operator (if any),

[4]

the left delimiter of the first block of the operation,

[5]

the text of the first block of the operation (that is, the contents of a quote, the regex of a match or substitution or the target list of a translation),

[6]

the right delimiter of the first block of the operation,

[7]

the left delimiter of the second block of the operation (that is, if it is a "s", "tr", or "y"),

[8]

the text of the second block of the operation (that is, the replacement of a substitution or the translation list of a translation),

[9]

the right delimiter of the second block of the operation (if any),

[10]

the trailing modifiers on the operation (if any).

For each of the fields marked "(if any)" the default value on success is an empty string. On failure, all of these values (except the remaining text) are "undef".

In a scalar context, "extract_quotelike" returns just the complete substring that matched a quotelike operation (or "undef" on failure). In a scalar or void context, the input text has the same substring (and any specified prefix) removed.

Examples:

        # Remove the first quotelike literal that appears in text
                $quotelike = extract_quotelike($text,'.*?');
        # Replace one or more leading whitespace-separated quotelike
        # literals in $_ with "<QLL>"
                do { $_ = join '<QLL>', (extract_quotelike)[2,1] } until $@;
        # Isolate the search pattern in a quotelike operation from $text
                ($op,$pat) = (extract_quotelike $text)[3,5];
                if ($op =~ /[ms]/)
                {
                        print "search pattern: $pat\n";
                }
                else
                {
                        print "$op is not a pattern matching operation\n";
                }

[0]

"<<'EOMSG'\nThis is the message.\nEOMSG\n" (i.e. the full extracted here document, including fore and aft delimiters),

[1]

" || die;\nexit;" (i.e. the remainder of the input text, concatenated),

[2]

"" (i.e. the prefix substring -- trivial in this case),

[3]

"<<" (i.e. the "name" of the quotelike operator)

[4]

"'EOMSG'" (i.e. the left delimiter of the here document, including any quotes),

[5]

"This is the message.\n" (i.e. the text of the here document),

[6]

"EOMSG" (i.e. the right delimiter of the here document),

[7..10]

"" (a here document has no second left delimiter, second text, second right delimiter, or trailing modifiers).

However, the matching position of the input variable would be set to "exit;" (i.e. after the closing delimiter of the here document), which would cause the earlier " || die;\nexit;" to be skipped in any sequence of code fragment extractions.

To avoid this problem, when it encounters a here document whilst extracting from a modifiable string, "extract_quotelike" silently rearranges the string to an equivalent piece of Perl:

        <<'EOMSG'
        This is the message.
        EOMSG
        || die;
        exit;

in which the here document is contiguous. It still leaves the matching position after the here document, but now the rest of the line on which the here document starts is not skipped.

To prevent <extract_quotelike> from mucking about with the input in this way (this is the only case where a list-context "extract_quotelike" does so), you can pass the input variable as an interpolated literal:

        $quotelike = extract_quotelike("$var");

1.

Try and match a closing delimiter bracket. If the bracket was the same species as the last opening bracket, return the substring to that point. If the bracket was mismatched, return an error.

2.

Try to match a quote or quotelike operator. If found, call "extract_quotelike" to eat it. If "extract_quotelike" fails, return the error it returned. Otherwise go back to step 1.

3.

Try to match an opening delimiter bracket. If found, call "extract_codeblock" recursively to eat the embedded block. If the recursive call fails, return an error. Otherwise, go back to step 1.

4.

Unconditionally match a bareword or any other single character, and then go back to step 1.

Examples:

        # Find a while loop in the text
                if ($text =~ s/.*?while\s*\{/{/)
                {
                        $loop = "while " . extract_codeblock($text);
                }
        # Remove the first round-bracketed list (which may include
        # round- or curly-bracketed code blocks or quotelike operators)
                extract_codeblock $text, "(){}", '[^(]*';

The ability to specify a different outermost delimiter bracket is useful in some circumstances. For example, in the Parse::RecDescent module, parser actions which are to be performed only on a successful parse are specified using a "<defer:...>" directive. For example:

        sentence: subject verb object
                        <defer: {$::theVerb = $item{verb}} >

Parse::RecDescent uses "extract_codeblock($text, '{}<>')" to extract the code within the "<defer:...>" directive, but there's a problem.

A deferred action like this:

                        <defer: {if ($count>10) {$count--}} >

will be incorrectly parsed as:

                        <defer: {if ($count>

because the "less than" operator is interpreted as a closing delimiter.

But, by extracting the directive using "extract_codeblock($text, '{}', undef, '<>')" the '>' character is only treated as a delimited at the outermost level of the code block, so the directive is parsed correctly.

1.

A string to be processed ($_ if the string is omitted or "undef")

2.

A reference to a list of subroutine references and/or qr// objects and/or literal strings and/or hash references, specifying the extractors to be used to split the string. If this argument is omitted (or "undef") the list:

        [
                sub { extract_variable($_[0], '') },
                sub { extract_quotelike($_[0],'') },
                sub { extract_codeblock($_[0],'{}','') },
        ]
    

is used.

3.

An number specifying the maximum number of fields to return. If this argument is omitted (or "undef"), split continues as long as possible.

If the third argument is N, then extraction continues until N fields have been successfully extracted, or until the string has been completely processed.

Note that in scalar and void contexts the value of this argument is automatically reset to 1 (under "-w", a warning is issued if the argument has to be reset).

4.

A value indicating whether unmatched substrings (see below) within the text should be skipped or returned as fields. If the value is true, such substrings are skipped. Otherwise, they are returned.

The extraction process works by applying each extractor in sequence to the text string.

If the extractor is a subroutine it is called in a list context and is expected to return a list of a single element, namely the extracted text. It may optionally also return two further arguments: a string representing the text left after extraction (like $' for a pattern match), and a string representing any prefix skipped before the extraction (like $` in a pattern match). Note that this is designed to facilitate the use of other Text::Balanced subroutines with "extract_multiple". Note too that the value returned by an extractor subroutine need not bear any relationship to the corresponding substring of the original text (see examples below).

If the extractor is a precompiled regular expression or a string, it is matched against the text in a scalar context with a leading '\G' and the gc modifiers enabled. The extracted value is either $1 if that variable is defined after the match, or else the complete match (i.e. $&).

If the extractor is a hash reference, it must contain exactly one element. The value of that element is one of the above extractor types (subroutine reference, regular expression, or string). The key of that element is the name of a class into which the successful return value of the extractor will be blessed.

If an extractor returns a defined value, that value is immediately treated as the next extracted field and pushed onto the list of fields. If the extractor was specified in a hash reference, the field is also blessed into the appropriate class,

If the extractor fails to match (in the case of a regex extractor), or returns an empty list or an undefined value (in the case of a subroutine extractor), it is assumed to have failed to extract. If none of the extractor subroutines succeeds, then one character is extracted from the start of the text and the extraction subroutines reapplied. Characters which are thus removed are accumulated and eventually become the next field (unless the fourth argument is true, in which case they are discarded).

For example, the following extracts substrings that are valid Perl variables:

        @fields = extract_multiple($text,
                                   [ sub { extract_variable($_[0]) } ],
                                   undef, 1);

This example separates a text into fields which are quote delimited, curly bracketed, and anything else. The delimited and bracketed parts are also blessed to identify them (the "anything else" is unblessed):

        @fields = extract_multiple($text,
                   [
                        { Delim => sub { extract_delimited($_[0],q{'"}) } },
                        { Brack => sub { extract_bracketed($_[0],'{}') } },
                   ]);

This call extracts the next single substring that is a valid Perl quotelike operator (and removes it from $text):

        $quotelike = extract_multiple($text,
                                      [
                                        sub { extract_quotelike($_[0]) },
                                      ], undef, 1);

Finally, here is yet another way to do comma-separated value parsing:

        $csv_text = "a,'x b',c";
        @fields = extract_multiple($csv_text,
                                  [
                                        sub { extract_delimited($_[0],q{'"}) },
                                        qr/([^,]+)/,
                                  ],
                                  undef,1);
        # @fields is now ('a', "'x b'", 'c')

The list in the second argument means: "Try and extract a ' or " delimited string, otherwise extract anything up to a comma...". The undef third argument means: "...as many times as possible...", and the true value in the fourth argument means "...discarding anything else that appears (i.e. the commas)".

If you wanted the commas preserved as separate fields (i.e. like split does if your split pattern has capturing parentheses), you would just make the last parameter undefined (or remove it).