Characters, Strings and Text Processing in Java

Size: px

Start display at page:

Download "Characters, Strings and Text Processing in Java"

Jasper Moore
5 years ago
Views:

1 Characters, Strings and Text Processing in Java Concepts: Characters, Code Points, Encodings Unicode: A to Z and more Jonas Kvarnström.liu.se 2012 Jonas Kvarnström.liu.se 2012 Characters 3 Character Repertoires 4 Step 1: A character Abstract definition: "An atom of information" Concrete examples: A B C D # "! ( ) å ä ö ċ õ ĕ ķ Ħ Ŗ ǽ Ґ ы Ω Ą Ж ΐ ਈਉਊਏਓਔਕਖਗਘਛ RM ریال 略 X Step 2: A character repertoire Just an unordered set of available characters ASCII: 128 characters, including 33 control characters ISO 646 SE: Like ASCII, but remove [ ]{\, add åäöåäö ISO Latin 1: Adds 32 control, 96 symbols (west Europe) ISO Latin 2: Adds 32 control, 96 symbols (east/central) Java 7 uses Unicode 6.0: Graphic characters: 109,242 Formatting: 142 Control: 65 Private Use: 137,468 Surrogate: 2,048 Non character: 66 Reserved: 865,081 More information:

Coded Character Sets 5 Coded Character Sets (2) 6 Step 3: A coded character set

= tab, 10 = line feed, 32 = space, 65 = A, 66 = B, ISO Latin 1: Like ASCII, with

j, Coded Character Sets (3) 7 Character Encodings 8 Unicode someexamples: 0000.

.0EDD LAO HO NO..LAO HO MO ໜໝༀ 2302..237A HOUSE..APL FUNCTIONAL SYMBOL ALPHA 2E80.

.CJK RADICAL RAP 33E0 IDEOGRAPHIC TELEGRAPH SYMBOL FOR DAY ONE 1039F UGARITIC

.<private-use-10fffd> Step 4: Character encoding We have a number for each

Most common encoding: UTF 8, variable 00000-0007F: 0xxxxxxx (same as ASCII!

2 Coded Character Sets 5 Coded Character Sets (2) 6 Step 3: A coded character set Assigns an integer for each character in the repertoire ASCII: values 0 to = tab, 10 = line feed, 32 = space, 65 = A, 66 = B, ISO Latin 1: Like ASCII, with additional at EBCDIC (IBM mainframes): 129 = a, 130 = b,, 137 = i, 145 = j, Coded Character Sets (3) 7 Character Encodings 8 Unicode someexamples: FF Like ISO Latin-1 03D0..03D6 GREEK BETA SYMBOL..GREEK PI SYMBOL 05B0..05B9 HEBREW POINT SHEVA..HEBREW POINT HOLAM 0A01 GURMUKHI SIGN ADAK BINDI ਈਉਊਏਓਔਕਖਗਘਛ 0EDC..0EDD LAO HO NO..LAO HO MO ໜໝༀ A HOUSE..APL FUNCTIONAL SYMBOL ALPHA 2E80..2E99 CJK RADICAL REPEAT..CJK RADICAL RAP 33E0 IDEOGRAPHIC TELEGRAPH SYMBOL FOR DAY ONE 1039F UGARITIC WORD DIVIDER FFFD <private-use >..<private-use-10fffd> Step 4: Character encoding We have a number for each character In a computer, this number must be coded into one or more bytes! At most 256 characters? Simple one byte per character Unicode: Won t even fit in two bytes! Most common encoding: UTF 8, variable F: 0xxxxxxx (same as ASCII!) FF: 110xxxxx 10xxxxxx FFFF: 1110xxxx 10xxxxxx 10xxxxxx FFFFF: 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx Efficient for English and similar languages Informative bit pattern! Example for UTF 8: Räksmörgås, 72 c3 a4 6b 73 6d c3 b c3 a5 73 In ISO Latin 1, this byte sequence would mean "rã ksmã rgã s"

Characters in Java 10 Characters Strings StringBuilders The char data type: 16 bits wide, storing Unicode characters from 0 to 65535 char c = 'x'; char c = 120; // Makes c an 'x', since Unicode

$Escape codes represent unprintable characters '\nnn': Character with octal value nnn ('\012' == character 10 = line feed) '\unnnn': Character with hex value nnnn ('\u03d0' == greek beta) '\b':$

3 Characters in Java 10 Characters Strings StringBuilders The char data type: 16 bits wide, storing Unicode characters from 0 to char c = 'x'; char c = 120; // Makes c an 'x', since Unicode character #120 is an 'x'. Escape codes represent unprintable characters '\nnn': Character with octal value nnn ('\012' == character 10 = line feed) '\unnnn': Character with hex value nnnn ('\u03d0' == greek beta) '\b': Backspace (ASCII 8) '\t': Tab (ASCII 9) '\n': Newline (ASCII 10) '\f': Form feed (ASCII 12) '\r': Carriage return (ASCII 13) '\'': Single quote (quote backslash quote quote) '\\': A backslash Jonas Kvarnström.liu.se 2012 The Character Class 11 Strings 1: Strings are s 12 Static utility methods in wrapper class java.lang.character Character types islowercase(), isuppercase(), istitlecase(), isdigit(), isletter(), isletterordigit(), iswhitespace(), Use these methods to accept greek lower case, kyrillic upper case, non arabic digits, and so on Character ch;...; if (ch.isletter()) {... Case conversion tolowercase(), touppercase(), totitlecase() The letter 'lj': Lower case 'lj'. Upper case 'LJ'. Title case 'Lj'. Strings in Java are objects, not primitive types or arrays str Immutable: Any modifications actually create new String objects String str = "Hello"; String literal in quotes escape codes can be used 5 ( 5) [0] H [1] e String pointer String object Character array used internally for storage Uses UTF-16 to support all of Unicode

Strings 2: Literals 13 Strings 3: Useful Methods 14 Even string literals represent objects Can be used directly, without being assigned to variables System.out.println("Hello".

$charat(1) 'b' Substring (end index is exclusive!) "abcdefgabc".substring(3,7) "defg" Without initial/trailing whitespace "\nfoo".trim() "foo" Split around regular expression "foo:bar::gazonk".$ split(":") {"foo", "bar", "", "gazonk" "foo:bar::gazonk".

split(":") {"foo", "bar", "", "gazonk" "foo:bar::gazonk".

4 Strings 2: Literals 13 Strings 3: Useful Methods 14 Even string literals represent objects Can be used directly, without being assigned to variables System.out.println("Hello".()); // 5 Represents a String object with 5 characters For every string literal in a class, a String object is interned Pre allocated in a global pool So never write new String("Hello"); you will just create an additional copy! Global Pool of String Literals 5 ( 5) [0] H [1] e String: Useful methods Length "foo".() 3 Specific character "abcdefgabc".charat(1) 'b' Substring (end index is exclusive!) "abcdefgabc".substring(3,7) "defg" Without initial/trailing whitespace "\nfoo".trim() "foo" Split around regular expression "foo:bar::gazonk".split(":") {"foo", "bar", "", "gazonk" "foo:bar::gazonk".split("o*") {"f", ":bar::gaz", "nk" and many more String conversions touppercase(), tolowercase() Strings 4: String Comparisons 15 Strings 5: String Comparisons 16 Comparing Strings equals(string other), equalsignorecase(string other) True if the strings are equal, false otherwise As usual, == tests whether they are stored in the same location! compareto(string other), comparetoignorecase(string other) 1 if this before other, 0 if equal, 1 if this after other Comparing Strings: == or equals()? String str1 = xy ; String str2 = x + y ; System.out.println(str1 == str2); System.out.println(str1.equals(str2)); // false // true str1 2 ( 2) [0] x [1] y str2 2 ( 2) [0] x [1] y

5 Strings 6: String Comparisons Different countries have different sorting rules Comparing á to a, sorting ß as ss, sorting ä as a with umlaut (German) or after å (Swedish) Use Collator, RuleBasedCollator for searching and sorting 17 // Uses system locale Collator collator = Collator.getInstance(); // Explicitly specify locale Locale sverige = Locale.forLanguageTag( sv-se ); Collator collator2 = Collator.getInstance(sverige); collator.compare( åäö", ÅÄÖ") // -1, 0, 1 Locale: Specifies language, country, Jonas Kvarnström.liu.se 2012 Formatting 1: Primitive to String 19 Formatting 2: to String 20 Convert primitive datatypes to text: Use the static String.valueOf() methods static String valueof(char c) { static String valueof(int i) { static String valueof(boolean b) { String str = String.valueOf(127); // "127" Every object has a string representation java.lang. has a tostring() method Returns a String representing the object System.out.println( o) calls o.tostring() Example: public class Circle { private double x, y, r; private Circle(double x, double y, double r) { this.x = x; this.y = y; this.r = r; public static void main(string[] args) { System.out.println(new Circle(2.0, 4.0, 5.0)); >> java Circle Circle@a5e17120

Formatting 3: to String 21 Formatting 4: String Concatenation 22 Why "Circle@a5e17120"?.toString() returns the class name + "@" + a unique ID The method should be overridden!

6 Formatting 3: to String 21 Formatting 4: String Concatenation 22 Why returns the class name + "@" + a unique ID The method should be overridden! Example: public class Circle { public String tostring() { return "Circle[" + x + ", " + y + ", " + r + ']'; public static void main(string[] args) { System.out.println(new Circle(2.0, 4.0, 5.0)); java Circle Circle[2.0, 4.0, 5.0] In the previous slide: "Circle[" + x + ", " + y + ", " + r + ']' Java overloads the + operator for string concatenation. How, when String objects are immutable? StringBuilder: Used for concatenating / manipulating char sequences Used implicitly for "+" on Strings translation by the Java compiler final StringBuilder buf = new StringBuilder(""); buf.append("circle["); buf.append(x); buf.append(", "); buf.append(y); buf.append(", "); buf.append(r); buf.append(']'); return buf.tostring(); Make a String with the same contents as the StringBuilder Formatting 5: Immutable Strings! 23 Formatting 6: StringBuilder Example 24 Constructing Strings in loops: Remember, String objects are immutable! public class ArrayList { private [] elements; private int size; public String tostring() { String result = "ArrayList<"; for (int i= 0; i< size; i++) { if (i!= 0) result = result + ", "; result = result + elements[i].tostring(); result = result + ">"; return result; Copy string into StringBuffer Append a few characters Copy StringBuffer into new String Discard old String and StringBuffer For efficiency, use StringBuilder directly (In loops and similar situations) public class ArrayList { private [] elements; private int size; public String tostring() { final StringBuilder buf = new StringBuilder(); buf.append("arraylist<"); for (int i = 0; i < size; i++) { if (i!= 0) buf.append(", "); buf.append(elements[i].tostring()); buf.append(">"); return buf.tostring(); Append a few characters Copy StringBuffer into new String Discard StringBuffer

7 Formatting 7: Detailed Control 25 Formatting 8: Detailed Control 26 For more detailed control over output: String.format() static method In PrintStream: printf() System.out.printf("Hello, %s!\n", person.name);// And so on String s1 = String.format("Hello, %s! You are %d years old.\n", person.name, person.age); // Minimum 30, maximum 40 characters (padded with spaces) String s2 = String.format("Hello, %30.40s!\n", person.name); For even more control over output, create a Formatter StringBuilder sb = new StringBuilder(); Formatter formatter = new Formatter(sb, Locale.US); // Here we specify an explicit locale String s3 = String.format(Locale.FRANCE, "e = %+10.4f", Math.E); // "e = +2,7183" // Advanced formatting: Use the same parameter multiple times Cadar c = new GregorianCadar(1995, MAY, 23); String s4 = String.format("Duke's Birthday: %1$tm %1$te,%1$tY", c); // s.equals("duke's Birthday: May 23, 1995") Parsing: String to Primitive 27 Convert text to primitive data: Use the corresponding wrapper class In class Integer: static int parseint(string str); In class Float: static float parsefloat(string str); In class Double: static double parsedouble(string str); float val = Float.parseFloat("127"); // 127.0f To split strings into "words", see also: StringTokenizer splits strings at given break characters (such as whitespace) String.split() splits strings using regular expression patterns StreamTokenizer for Java like languages (parses comments, ) BreakIterator for natural languagetext

Assoc. Prof. Marenglen Biba. (C) 2010 Pearson Education, Inc. All rights reserved.

Assoc. Prof. Marenglen Biba. (C) 2010 Pearson Education, Inc. All rights reserved. Assoc. Prof. Marenglen Biba (C) 2010 Pearson Education, Inc. All Advanced Java This chapter discusses class String, class StringBuilder and class Character from the java.lang package. These classes provide