- java.lang.Object
-
- com.google.gson.stream.JsonReader
-
- All Implemented Interfaces:
java.io.Closeable
,java.lang.AutoCloseable
- Direct Known Subclasses:
JsonTreeReader
public class JsonReader extends java.lang.Object implements java.io.Closeable
Reads a JSON (RFC 7159) encoded value as a stream of tokens. This stream includes both literal values (strings, numbers, booleans, and nulls) as well as the begin and end delimiters of objects and arrays. The tokens are traversed in depth-first order, the same order that they appear in the JSON document. Within JSON objects, name/value pairs are represented by a single token.Parsing JSON
To create a recursive descent parser for your own JSON streams, first create an entry point method that creates aJsonReader
.Next, create handler methods for each structure in your JSON text. You'll need a method for each object type and for each array type.
- Within array handling methods, first call
beginArray()
to consume the array's opening bracket. Then create a while loop that accumulates values, terminating whenhasNext()
is false. Finally, read the array's closing bracket by callingendArray()
. - Within object handling methods, first call
beginObject()
to consume the object's opening brace. Then create a while loop that assigns values to local variables based on their name. This loop should terminate whenhasNext()
is false. Finally, read the object's closing brace by callingendObject()
.
When a nested object or array is encountered, delegate to the corresponding handler method.
When an unknown name is encountered, strict parsers should fail with an exception. Lenient parsers should call
skipValue()
to recursively skip the value's nested tokens, which may otherwise conflict.If a value may be null, you should first check using
peek()
. Null literals can be consumed using eithernextNull()
orskipValue()
.Example
Suppose we'd like to parse a stream of messages such as the following:[ { "id": 912345678901, "text": "How do I read a JSON stream in Java?", "geo": null, "user": { "name": "json_newb", "followers_count": 41 } }, { "id": 912345678902, "text": "@json_newb just use JsonReader!", "geo": [50.454722, -104.606667], "user": { "name": "jesse", "followers_count": 2 } } ]
public List<Message> readJsonStream(InputStream in) throws IOException { JsonReader reader = new JsonReader(new InputStreamReader(in, "UTF-8")); try { return readMessagesArray(reader); } finally { reader.close(); } } public List<Message> readMessagesArray(JsonReader reader) throws IOException { List<Message> messages = new ArrayList<>(); reader.beginArray(); while (reader.hasNext()) { messages.add(readMessage(reader)); } reader.endArray(); return messages; } public Message readMessage(JsonReader reader) throws IOException { long id = -1; String text = null; User user = null; List<Double> geo = null; reader.beginObject(); while (reader.hasNext()) { String name = reader.nextName(); if (name.equals("id")) { id = reader.nextLong(); } else if (name.equals("text")) { text = reader.nextString(); } else if (name.equals("geo") && reader.peek() != JsonToken.NULL) { geo = readDoublesArray(reader); } else if (name.equals("user")) { user = readUser(reader); } else { reader.skipValue(); } } reader.endObject(); return new Message(id, text, user, geo); } public List<Double> readDoublesArray(JsonReader reader) throws IOException { List<Double> doubles = new ArrayList<>(); reader.beginArray(); while (reader.hasNext()) { doubles.add(reader.nextDouble()); } reader.endArray(); return doubles; } public User readUser(JsonReader reader) throws IOException { String username = null; int followersCount = -1; reader.beginObject(); while (reader.hasNext()) { String name = reader.nextName(); if (name.equals("name")) { username = reader.nextString(); } else if (name.equals("followers_count")) { followersCount = reader.nextInt(); } else { reader.skipValue(); } } reader.endObject(); return new User(username, followersCount); }
Number Handling
This reader permits numeric values to be read as strings and string values to be read as numbers. For example, both elements of the JSON array[1, "1"]
may be read using eithernextInt()
ornextString()
. This behavior is intended to prevent lossy numeric conversions: double is JavaScript's only numeric type and very large values like9007199254740993
cannot be represented exactly on that platform. To minimize precision loss, extremely large values should be written and read as strings in JSON.Non-Execute Prefix
Web servers that serve private data using JSON may be vulnerable to Cross-site request forgery attacks. In such an attack, a malicious site gains access to a private JSON file by executing it with an HTML<script>
tag.Prefixing JSON files with
")]}'\n"
makes them non-executable by<script>
tags, disarming the attack. Since the prefix is malformed JSON, strict parsing fails when it is encountered. This class permits the non-execute prefix whenlenient parsing
is enabled.Each
JsonReader
may be used to read a single JSON stream. Instances of this class are not thread safe.- Since:
- 1.6
-
-
Field Summary
Fields Modifier and Type Field Description private char[]
buffer
Use a manual buffer to easily read and unread upcoming characters, and also so we can create strings without an intermediate StringBuilder.(package private) static int
BUFFER_SIZE
private java.io.Reader
in
The input JSON.private boolean
lenient
True to accept non-spec compliant JSONprivate int
limit
private int
lineNumber
private int
lineStart
private static long
MIN_INCOMPLETE_INTEGER
private static int
NUMBER_CHAR_DECIMAL
private static int
NUMBER_CHAR_DIGIT
private static int
NUMBER_CHAR_EXP_DIGIT
private static int
NUMBER_CHAR_EXP_E
private static int
NUMBER_CHAR_EXP_SIGN
private static int
NUMBER_CHAR_FRACTION_DIGIT
private static int
NUMBER_CHAR_NONE
private static int
NUMBER_CHAR_SIGN
private int[]
pathIndices
private java.lang.String[]
pathNames
(package private) int
peeked
private static int
PEEKED_BEGIN_ARRAY
private static int
PEEKED_BEGIN_OBJECT
private static int
PEEKED_BUFFERED
When this is returned, the string value is stored in peekedString.private static int
PEEKED_DOUBLE_QUOTED
private static int
PEEKED_DOUBLE_QUOTED_NAME
private static int
PEEKED_END_ARRAY
private static int
PEEKED_END_OBJECT
private static int
PEEKED_EOF
private static int
PEEKED_FALSE
private static int
PEEKED_LONG
When this is returned, the integer value is stored in peekedLong.private static int
PEEKED_NONE
private static int
PEEKED_NULL
private static int
PEEKED_NUMBER
private static int
PEEKED_SINGLE_QUOTED
private static int
PEEKED_SINGLE_QUOTED_NAME
private static int
PEEKED_TRUE
private static int
PEEKED_UNQUOTED
private static int
PEEKED_UNQUOTED_NAME
private long
peekedLong
A peeked value that was composed entirely of digits with an optional leading dash.private int
peekedNumberLength
The number of characters in a peeked number literal.private java.lang.String
peekedString
A peeked string that should be parsed on the next double, long or string.private int
pos
private int[]
stack
private int
stackSize
-
Constructor Summary
Constructors Constructor Description JsonReader(java.io.Reader in)
Creates a new instance that reads a JSON-encoded stream fromin
.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description void
beginArray()
Consumes the next token from the JSON stream and asserts that it is the beginning of a new array.void
beginObject()
Consumes the next token from the JSON stream and asserts that it is the beginning of a new object.private void
checkLenient()
void
close()
Closes this JSON reader and the underlyingReader
.private void
consumeNonExecutePrefix()
Consumes the non-execute prefix if it exists.(package private) int
doPeek()
void
endArray()
Consumes the next token from the JSON stream and asserts that it is the end of the current array.void
endObject()
Consumes the next token from the JSON stream and asserts that it is the end of the current object.private boolean
fillBuffer(int minimum)
Returns true oncelimit - pos >= minimum
.java.lang.String
getPath()
Returns a JsonPath in dot-notation to the next (or current) location in the JSON document: For JSON arrays the path points to the index of the next element (even if there are no further elements). For JSON objects the path points to the last property, or to the current property if its value has not been consumed yet.private java.lang.String
getPath(boolean usePreviousPath)
java.lang.String
getPreviousPath()
Returns a JsonPath in dot-notation to the previous (or current) location in the JSON document: For JSON arrays the path points to the index of the previous element.
If no element has been consumed yet it uses the index 0 (even if there are no elements). For JSON objects the path points to the last property, or to the current property if its value has not been consumed yet.boolean
hasNext()
Returns true if the current array or object has another element.boolean
isLenient()
Returns true if this parser is liberal in what it accepts.private boolean
isLiteral(char c)
(package private) java.lang.String
locationString()
boolean
nextBoolean()
Returns theboolean
value of the next token, consuming it.double
nextDouble()
Returns thedouble
value of the next token, consuming it.int
nextInt()
Returns theint
value of the next token, consuming it.long
nextLong()
Returns thelong
value of the next token, consuming it.java.lang.String
nextName()
Returns the next token, aproperty name
, and consumes it.private int
nextNonWhitespace(boolean throwOnEof)
Returns the next character in the stream that is neither whitespace nor a part of a comment.void
nextNull()
Consumes the next token from the JSON stream and asserts that it is a literal null.private java.lang.String
nextQuotedValue(char quote)
Returns the string up to but not includingquote
, unescaping any character escape sequences encountered along the way.java.lang.String
nextString()
Returns thestring
value of the next token, consuming it.private java.lang.String
nextUnquotedValue()
Returns an unquoted value as a string.JsonToken
peek()
Returns the type of the next token without consuming it.private int
peekKeyword()
private int
peekNumber()
private void
push(int newTop)
private char
readEscapeCharacter()
Unescapes the character identified by the character or characters that immediately follow a backslash.void
setLenient(boolean lenient)
Configure this parser to be liberal in what it accepts.private void
skipQuotedValue(char quote)
private boolean
skipTo(java.lang.String toFind)
private void
skipToEndOfLine()
Advances the position until after the next newline character.private void
skipUnquotedValue()
void
skipValue()
Skips the next value recursively.private java.io.IOException
syntaxError(java.lang.String message)
Throws a new IO exception with the given message and a context snippet with this reader's content.java.lang.String
toString()
-
-
-
Field Detail
-
MIN_INCOMPLETE_INTEGER
private static final long MIN_INCOMPLETE_INTEGER
- See Also:
- Constant Field Values
-
PEEKED_NONE
private static final int PEEKED_NONE
- See Also:
- Constant Field Values
-
PEEKED_BEGIN_OBJECT
private static final int PEEKED_BEGIN_OBJECT
- See Also:
- Constant Field Values
-
PEEKED_END_OBJECT
private static final int PEEKED_END_OBJECT
- See Also:
- Constant Field Values
-
PEEKED_BEGIN_ARRAY
private static final int PEEKED_BEGIN_ARRAY
- See Also:
- Constant Field Values
-
PEEKED_END_ARRAY
private static final int PEEKED_END_ARRAY
- See Also:
- Constant Field Values
-
PEEKED_TRUE
private static final int PEEKED_TRUE
- See Also:
- Constant Field Values
-
PEEKED_FALSE
private static final int PEEKED_FALSE
- See Also:
- Constant Field Values
-
PEEKED_NULL
private static final int PEEKED_NULL
- See Also:
- Constant Field Values
-
PEEKED_SINGLE_QUOTED
private static final int PEEKED_SINGLE_QUOTED
- See Also:
- Constant Field Values
-
PEEKED_DOUBLE_QUOTED
private static final int PEEKED_DOUBLE_QUOTED
- See Also:
- Constant Field Values
-
PEEKED_UNQUOTED
private static final int PEEKED_UNQUOTED
- See Also:
- Constant Field Values
-
PEEKED_BUFFERED
private static final int PEEKED_BUFFERED
When this is returned, the string value is stored in peekedString.- See Also:
- Constant Field Values
-
PEEKED_SINGLE_QUOTED_NAME
private static final int PEEKED_SINGLE_QUOTED_NAME
- See Also:
- Constant Field Values
-
PEEKED_DOUBLE_QUOTED_NAME
private static final int PEEKED_DOUBLE_QUOTED_NAME
- See Also:
- Constant Field Values
-
PEEKED_UNQUOTED_NAME
private static final int PEEKED_UNQUOTED_NAME
- See Also:
- Constant Field Values
-
PEEKED_LONG
private static final int PEEKED_LONG
When this is returned, the integer value is stored in peekedLong.- See Also:
- Constant Field Values
-
PEEKED_NUMBER
private static final int PEEKED_NUMBER
- See Also:
- Constant Field Values
-
PEEKED_EOF
private static final int PEEKED_EOF
- See Also:
- Constant Field Values
-
NUMBER_CHAR_NONE
private static final int NUMBER_CHAR_NONE
- See Also:
- Constant Field Values
-
NUMBER_CHAR_SIGN
private static final int NUMBER_CHAR_SIGN
- See Also:
- Constant Field Values
-
NUMBER_CHAR_DIGIT
private static final int NUMBER_CHAR_DIGIT
- See Also:
- Constant Field Values
-
NUMBER_CHAR_DECIMAL
private static final int NUMBER_CHAR_DECIMAL
- See Also:
- Constant Field Values
-
NUMBER_CHAR_FRACTION_DIGIT
private static final int NUMBER_CHAR_FRACTION_DIGIT
- See Also:
- Constant Field Values
-
NUMBER_CHAR_EXP_E
private static final int NUMBER_CHAR_EXP_E
- See Also:
- Constant Field Values
-
NUMBER_CHAR_EXP_SIGN
private static final int NUMBER_CHAR_EXP_SIGN
- See Also:
- Constant Field Values
-
NUMBER_CHAR_EXP_DIGIT
private static final int NUMBER_CHAR_EXP_DIGIT
- See Also:
- Constant Field Values
-
in
private final java.io.Reader in
The input JSON.
-
lenient
private boolean lenient
True to accept non-spec compliant JSON
-
BUFFER_SIZE
static final int BUFFER_SIZE
- See Also:
- Constant Field Values
-
buffer
private final char[] buffer
Use a manual buffer to easily read and unread upcoming characters, and also so we can create strings without an intermediate StringBuilder. We decode literals directly out of this buffer, so it must be at least as long as the longest token that can be reported as a number.
-
pos
private int pos
-
limit
private int limit
-
lineNumber
private int lineNumber
-
lineStart
private int lineStart
-
peeked
int peeked
-
peekedLong
private long peekedLong
A peeked value that was composed entirely of digits with an optional leading dash. Positive values may not have a leading 0.
-
peekedNumberLength
private int peekedNumberLength
The number of characters in a peeked number literal. Increment 'pos' by this after reading a number.
-
peekedString
private java.lang.String peekedString
A peeked string that should be parsed on the next double, long or string. This is populated before a numeric value is parsed and used if that parsing fails.
-
stack
private int[] stack
-
stackSize
private int stackSize
-
pathNames
private java.lang.String[] pathNames
-
pathIndices
private int[] pathIndices
-
-
Method Detail
-
setLenient
public final void setLenient(boolean lenient)
Configure this parser to be liberal in what it accepts. By default, this parser is strict and only accepts JSON as specified by RFC 4627. Setting the parser to lenient causes it to ignore the following syntax errors:- Streams that start with the non-execute
prefix,
")]}'\n"
. - Streams that include multiple top-level values. With strict parsing, each stream must contain exactly one top-level value.
- Numbers may be
NaNs
orinfinities
. - End of line comments starting with
//
or#
and ending with a newline character. - C-style comments starting with
/*
and ending with*
/
. Such comments may not be nested. - Names that are unquoted or
'single quoted'
. - Strings that are unquoted or
'single quoted'
. - Array elements separated by
;
instead of,
. - Unnecessary array separators. These are interpreted as if null was the omitted value.
- Names and values separated by
=
or=>
instead of:
. - Name/value pairs separated by
;
instead of,
.
Note: Even in strict mode there are slight derivations from the JSON specification:
- JsonReader allows the literals
true
,false
andnull
to have any capitalization, for examplefAlSe
- JsonReader supports the escape sequence
\'
, representing a'
- JsonReader supports the escape sequence
\LF
(withLF
being the Unicode character U+000A), resulting in aLF
within the read JSON string - JsonReader allows unescaped control characters (U+0000 through U+001F)
- Streams that start with the non-execute
prefix,
-
isLenient
public final boolean isLenient()
Returns true if this parser is liberal in what it accepts.
-
beginArray
public void beginArray() throws java.io.IOException
Consumes the next token from the JSON stream and asserts that it is the beginning of a new array.- Throws:
java.io.IOException
-
endArray
public void endArray() throws java.io.IOException
Consumes the next token from the JSON stream and asserts that it is the end of the current array.- Throws:
java.io.IOException
-
beginObject
public void beginObject() throws java.io.IOException
Consumes the next token from the JSON stream and asserts that it is the beginning of a new object.- Throws:
java.io.IOException
-
endObject
public void endObject() throws java.io.IOException
Consumes the next token from the JSON stream and asserts that it is the end of the current object.- Throws:
java.io.IOException
-
hasNext
public boolean hasNext() throws java.io.IOException
Returns true if the current array or object has another element.- Throws:
java.io.IOException
-
peek
public JsonToken peek() throws java.io.IOException
Returns the type of the next token without consuming it.- Throws:
java.io.IOException
-
doPeek
int doPeek() throws java.io.IOException
- Throws:
java.io.IOException
-
peekKeyword
private int peekKeyword() throws java.io.IOException
- Throws:
java.io.IOException
-
peekNumber
private int peekNumber() throws java.io.IOException
- Throws:
java.io.IOException
-
isLiteral
private boolean isLiteral(char c) throws java.io.IOException
- Throws:
java.io.IOException
-
nextName
public java.lang.String nextName() throws java.io.IOException
Returns the next token, aproperty name
, and consumes it.- Throws:
java.io.IOException
- if the next token in the stream is not a property name.
-
nextString
public java.lang.String nextString() throws java.io.IOException
Returns thestring
value of the next token, consuming it. If the next token is a number, this method will return its string form.- Throws:
java.lang.IllegalStateException
- if the next token is not a string or if this reader is closed.java.io.IOException
-
nextBoolean
public boolean nextBoolean() throws java.io.IOException
Returns theboolean
value of the next token, consuming it.- Throws:
java.lang.IllegalStateException
- if the next token is not a boolean or if this reader is closed.java.io.IOException
-
nextNull
public void nextNull() throws java.io.IOException
Consumes the next token from the JSON stream and asserts that it is a literal null.- Throws:
java.lang.IllegalStateException
- if the next token is not null or if this reader is closed.java.io.IOException
-
nextDouble
public double nextDouble() throws java.io.IOException
Returns thedouble
value of the next token, consuming it. If the next token is a string, this method will attempt to parse it as a double usingDouble.parseDouble(String)
.- Throws:
java.lang.IllegalStateException
- if the next token is not a literal value.java.lang.NumberFormatException
- if the next literal value cannot be parsed as a double, or is non-finite.java.io.IOException
-
nextLong
public long nextLong() throws java.io.IOException
Returns thelong
value of the next token, consuming it. If the next token is a string, this method will attempt to parse it as a long. If the next token's numeric value cannot be exactly represented by a Javalong
, this method throws.- Throws:
java.lang.IllegalStateException
- if the next token is not a literal value.java.lang.NumberFormatException
- if the next literal value cannot be parsed as a number, or exactly represented as a long.java.io.IOException
-
nextQuotedValue
private java.lang.String nextQuotedValue(char quote) throws java.io.IOException
Returns the string up to but not includingquote
, unescaping any character escape sequences encountered along the way. The opening quote should have already been read. This consumes the closing quote, but does not include it in the returned string.- Parameters:
quote
- either ' or ".- Throws:
java.lang.NumberFormatException
- if any unicode escape sequences are malformed.java.io.IOException
-
nextUnquotedValue
private java.lang.String nextUnquotedValue() throws java.io.IOException
Returns an unquoted value as a string.- Throws:
java.io.IOException
-
skipQuotedValue
private void skipQuotedValue(char quote) throws java.io.IOException
- Throws:
java.io.IOException
-
skipUnquotedValue
private void skipUnquotedValue() throws java.io.IOException
- Throws:
java.io.IOException
-
nextInt
public int nextInt() throws java.io.IOException
Returns theint
value of the next token, consuming it. If the next token is a string, this method will attempt to parse it as an int. If the next token's numeric value cannot be exactly represented by a Javaint
, this method throws.- Throws:
java.lang.IllegalStateException
- if the next token is not a literal value.java.lang.NumberFormatException
- if the next literal value cannot be parsed as a number, or exactly represented as an int.java.io.IOException
-
close
public void close() throws java.io.IOException
Closes this JSON reader and the underlyingReader
.- Specified by:
close
in interfacejava.lang.AutoCloseable
- Specified by:
close
in interfacejava.io.Closeable
- Throws:
java.io.IOException
-
skipValue
public void skipValue() throws java.io.IOException
Skips the next value recursively. If it is an object or array, all nested elements are skipped. This method is intended for use when the JSON token stream contains unrecognized or unhandled values.- Throws:
java.io.IOException
-
push
private void push(int newTop)
-
fillBuffer
private boolean fillBuffer(int minimum) throws java.io.IOException
Returns true oncelimit - pos >= minimum
. If the data is exhausted before that many characters are available, this returns false.- Throws:
java.io.IOException
-
nextNonWhitespace
private int nextNonWhitespace(boolean throwOnEof) throws java.io.IOException
Returns the next character in the stream that is neither whitespace nor a part of a comment. When this returns, the returned character is always atbuffer[pos-1]
; this means the caller can always push back the returned character by decrementingpos
.- Throws:
java.io.IOException
-
checkLenient
private void checkLenient() throws java.io.IOException
- Throws:
java.io.IOException
-
skipToEndOfLine
private void skipToEndOfLine() throws java.io.IOException
Advances the position until after the next newline character. If the line is terminated by "\r\n", the '\n' must be consumed as whitespace by the caller.- Throws:
java.io.IOException
-
skipTo
private boolean skipTo(java.lang.String toFind) throws java.io.IOException
- Parameters:
toFind
- a string to search for. Must not contain a newline.- Throws:
java.io.IOException
-
toString
public java.lang.String toString()
- Overrides:
toString
in classjava.lang.Object
-
locationString
java.lang.String locationString()
-
getPath
private java.lang.String getPath(boolean usePreviousPath)
-
getPreviousPath
public java.lang.String getPreviousPath()
Returns a JsonPath in dot-notation to the previous (or current) location in the JSON document:- For JSON arrays the path points to the index of the previous element.
If no element has been consumed yet it uses the index 0 (even if there are no elements). - For JSON objects the path points to the last property, or to the current property if its value has not been consumed yet.
This method can be useful to add additional context to exception messages after a value has been consumed.
- For JSON arrays the path points to the index of the previous element.
-
getPath
public java.lang.String getPath()
Returns a JsonPath in dot-notation to the next (or current) location in the JSON document:- For JSON arrays the path points to the index of the next element (even if there are no further elements).
- For JSON objects the path points to the last property, or to the current property if its value has not been consumed yet.
This method can be useful to add additional context to exception messages before a value is consumed, for example when the peeked token is unexpected.
-
readEscapeCharacter
private char readEscapeCharacter() throws java.io.IOException
Unescapes the character identified by the character or characters that immediately follow a backslash. The backslash '\' should have already been read. This supports both unicode escapes "u000A" and two-character escapes "\n".- Throws:
java.lang.NumberFormatException
- if any unicode escape sequences are malformed.java.io.IOException
-
syntaxError
private java.io.IOException syntaxError(java.lang.String message) throws java.io.IOException
Throws a new IO exception with the given message and a context snippet with this reader's content.- Throws:
java.io.IOException
-
consumeNonExecutePrefix
private void consumeNonExecutePrefix() throws java.io.IOException
Consumes the non-execute prefix if it exists.- Throws:
java.io.IOException
-
-