Tokenize the string to string array in Java

Write a function that takes a string and delimiter as input and return the array as output. Output array contain the tokens. Function should be flexible enough, where user can choose the option to trim the tokens and ignore empty tokens.

 

Signature

public static String[] tokenize(final String str, final String delimiter, final boolean trimTokens, final boolean ignoreEmptyTokens)

 

Find the below working application.

 

StringTokenizerDemo.java

package com.sample.app.strings;

import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.StringTokenizer;

public class StringTokenizerDemo {

	/**
	 * Tokenize the given String into a String array.
	 * 
	 * @param str               input string
	 * @param delimiter
	 * @param trimTokens        trim the tokens if set to true.
	 * @param ignoreEmptyTokens Ignore empty tokens, if set to true.
	 * @return null, if the input string is null, else tokenize the given string
	 *         with given delimiter and return the tokens as an array.
	 */
	public static String[] tokenize(final String str, final String delimiter, final boolean trimTokens,
			final boolean ignoreEmptyTokens) {

		if (str == null) {
			return null;
		}

		if (delimiter == null || delimiter.isEmpty()) {
			return new String[] { str };
		}

		final StringTokenizer stringTokenizer = new StringTokenizer(str, delimiter);
		final List<String> tokens = new ArrayList<>();

		while (stringTokenizer.hasMoreTokens()) {
			String token = stringTokenizer.nextToken();

			if (trimTokens) {
				token = token.trim();
			}

			if (ignoreEmptyTokens && token.length() == 0) {
				continue;
			}
			
			tokens.add(token);
		}
		return tokens.toArray(new String[0]);
	}

	public static void main(String[] args) {
		final String message = "Hi there,    ,how are you";
		final String[] tokens1 = tokenize(message, ",", true, true);
		final String[] tokens2 = tokenize(message, ",", false, true);
		final String[] tokens3 = tokenize(message, ",", true, false);

		System.out.println(Arrays.asList(tokens1));
		System.out.println(Arrays.asList(tokens2));
		System.out.println(Arrays.asList(tokens3));

	}
}

 

Output

[Hi there, how are you]
[Hi there,     , how are you]
[Hi there, , how are you]

 

Previous                                                 Next                                                 Home